Something Fundamental Has Changed
The first time an AI agent opens a pull request that gets merged without a human reviewing the actual code, something fundamental changes. Not because the code is necessarily bad — it might be perfectly correct. But because the assumptions baked into every CI/CD pipeline ever built suddenly no longer hold.
Every gate, every policy, every test in your pipeline was designed around one implicit assumption: a human wrote this code with conscious intent. They understood what they were building, they knew what the adjacent systems do, and they could be held accountable if it caused a production incident.
AI agents break that assumption completely. And the software industry has not yet reckoned with what that means for the infrastructure we use to ship software.
What AI Agents Are Actually Doing Today
This is not a future scenario. It is the current state.
Tools like GitHub Copilot Workspace, Devin, Cursor in agentic mode, and Claude Code agents are already:
- Writing complete features based on a natural language description in a GitHub issue, then opening a pull request with tests included
- Fixing failing CI jobs autonomously — reading the error output, modifying the code, and pushing the fix
- Updating dependencies across codebases — bumping packages, resolving breaking changes, and running the test suite
- Writing Terraform and Kubernetes configurations from architectural descriptions, making infrastructure changes at the same speed as application changes
- Responding to on-call alerts by analyzing logs, identifying probable root causes, and applying known remediation patterns
Each of these is useful. Each of them is also a new attack surface, a new source of subtle correctness failures, and a new challenge for every process your organization has built around the assumption of human authorship.
The Trust Problem
When a human developer opens a pull request, the review process is fundamentally about trust and communication. The reviewer asks: does this code do what the author intended, does the intention match the requirement, and is the approach sound? If something goes wrong, there is a person who can explain what they were thinking and learn from the failure.
AI agents collapse this model in two important ways.
First, they are non-deterministic. The same prompt, on the same codebase, on the same day, can produce meaningfully different code. Your test suite might pass on one run and fail on another not because the tests are flaky — but because the agent took a different approach to the same problem. Traditional CI pipelines have no concept of “the author might have meant something different.”
Second, they have no ground truth about intent. A human reviewer can ask “why did you do it this way?” and get a meaningful answer. When you ask an AI agent, you get a post-hoc rationalization generated from the same model that wrote the code. The reasoning and the code are not independent — they come from the same source, which means the reasoning cannot reliably identify errors in the code.
This is not a reason to stop using AI agents. It is a reason to think very carefully about what your pipeline needs to look like when they are involved.
CI/CD as the Governance Layer
The insight that changes everything: when AI agents are in the loop, the CI/CD pipeline stops being a quality gate and becomes a governance layer.
In a purely human development workflow, the pipeline validates what has already been reviewed. Pull request review is the primary trust checkpoint; CI is a mechanical verification that the human-reviewed code is internally consistent.
When AI agents are committing code, this ordering inverts. The agent may have written code that passes every automated check but violates an architectural invariant, introduces a subtle security flaw, or implements functionality that technically satisfies the ticket description but contradicts the product intent. The pipeline must now catch things that only emerge from the combination of what the agent did and what the broader system does — things that human review previously caught through context and judgment.
The pipeline needs new stages. Not instead of existing ones — in addition to them.
What an AI-Aware Pipeline Looks Like
Provenance and Attribution
Every AI-generated change must be tagged at commit time with metadata that survives into production: which model produced it, which prompt triggered it, what context it had access to, and what human (if any) authorized the action.
This is not optional for regulated industries or any organization that needs to answer “how did this get into production?” after an incident. SLSA (Supply-chain Levels for Software Artifacts) provides a framework for this. Signed commits via Sigstore/Gitsign provide the cryptographic chain. The infrastructure exists — it simply needs to be required for AI-authored changes the same way it is required for release builds.
AI-Specific Static Analysis
Standard SAST tools were not designed to catch the failure modes of AI-generated code. They look for known vulnerability patterns. AI agents can produce novel patterns that are semantically incorrect without triggering any known rule.
Emerging tooling addresses this gap:
- Semgrep rules tuned for AI output patterns — particularly around hallucinated API calls, incorrect error handling propagation, and subtle race conditions that LLMs reproduce at higher rates than human developers
- CodeQL deep semantic analysis that catches correctness issues beyond surface-level patterns
- Custom policy rules (OPA/Conftest) that encode architectural invariants — if an AI agent is modifying infrastructure code, it should not be able to create resources outside the approved module library without a specific human override
Hallucinated Dependency Detection
This is one of the most underappreciated risks of AI-generated code. LLMs occasionally invent package names that do not exist — names that are plausible given the context, syntactically valid, and sometimes similar to real packages. If a malicious actor registers that package name (a technique called dependency confusion or typosquatting), any pipeline that installs dependencies without strict lockfile enforcement will fetch and execute attacker-controlled code.
The mitigation is not complicated: enforce lockfile usage (no pip install without requirements.txt, no npm install without package-lock.json), validate every new dependency against known-good registries before allowing it into the dependency graph, and treat any unrecognized dependency in an AI-authored PR as a required human review item regardless of other checks passing.
Prompt Injection Audit
AI coding agents read issue descriptions, PR comments, code comments, and sometimes external documentation to build context for their actions. Each of these is a potential injection point for an attacker to influence the agent’s behavior.
A carefully crafted GitHub issue comment or a malicious string embedded in a third-party library’s documentation could instruct an agent to add a backdoor, exfiltrate environment variables, or disable security controls — while the agent genuinely “believes” it is following legitimate instructions.
Prompt injection is not hypothetical. It has been demonstrated against multiple major AI coding tools. Every input surface that an AI agent reads should be treated as untrusted input, and the agent’s actions should be bounded by the principle of least privilege: an agent fixing a bug in the authentication module should not have the permissions to modify the CI pipeline configuration.
The Human Review That Actually Matters
When AI agents are involved, human review changes character. It is no longer primarily about whether the code is correct — the tests handle that better than a reviewer skimming a diff. It becomes a question of intent alignment: does this change reflect what the organization actually wants, in a way that the organization will be comfortable with when it reaches production?
The reviewer is not checking the implementation. They are checking the decision. Should this feature exist in this form? Does this infrastructure change reflect our architectural direction? Is this the right place in the codebase for this logic?
This is harder than traditional code review in some ways and easier in others. It requires strong product and architectural context but less time on mechanical correctness checking. The review queue changes shape — fewer reviews that take longer and require more senior judgment, rather than more reviews that are faster and more routine.
The GitOps Connection
If you have read our post on GitOps, the connection to AI agents is direct and significant.
GitOps makes Git the single source of truth for infrastructure state. Every change is versioned, reviewable, and reversible. When a human engineer makes a change, this audit trail is valuable. When an AI agent makes infrastructure changes — spinning up resources, modifying network policies, updating Kubernetes manifests — this audit trail becomes essential.
Without GitOps, an AI agent with cloud credentials can make infrastructure changes that are invisible to the team until something breaks. With GitOps, every change the agent makes is a pull request in Git, reviewed by the same policy and human gates as human-authored changes. The agent cannot deploy anything that has not passed through the pipeline.
This is not a constraint on AI agent capability. It is the governance structure that makes AI agent capability safe to use at scale. An agent that can do anything is not useful in production — it is a liability. An agent that can do anything that passes your policy gates and gets human sign-off is enormously useful, and the organization can confidently extend its autonomy over time as trust is established.
The same logic applies to the Policy as Code principles we have covered. When AI agents are writing Terraform or Kubernetes configurations, the OPA and Checkov gates in your pipeline are not checking human-authored infrastructure code anymore. They are the last automated defense before an AI-generated configuration reaches your cloud account. That changes their priority from “useful” to “non-negotiable.”
The Specific Risks You Need to Model
Organizations deploying AI agents in their pipelines should explicitly model the following threat scenarios:
1. The Compromised Agent An AI agent whose behavior has been manipulated through prompt injection or a compromised model endpoint. It produces code that appears correct but contains a subtle backdoor or data exfiltration path. Mitigation: treat AI-generated code as untrusted until it passes all pipeline gates, and ensure agents operate with minimal permissions.
2. The Confident Hallucination An agent that generates code using an API that does not exist, or that implements a security algorithm incorrectly while producing confident, readable documentation for the implementation. Mitigation: semantic analysis beyond surface-level testing, mandatory human review for any security-sensitive changes regardless of test results.
3. The Runaway Agent An agent given broad permissions and an underspecified task that makes a series of individually reasonable decisions leading to a collectively bad outcome — deleting old infrastructure that is still in use, overwriting production secrets, or triggering a mass refactor that breaks dozens of downstream services. Mitigation: minimal permission scope, mandatory change set size limits, and breakpoints that require human confirmation before proceeding beyond defined thresholds.
4. The Supply Chain Substitution An agent that, through hallucination or manipulation, introduces a dependency on a package that looks legitimate but is attacker-controlled. Mitigation: strict lockfile enforcement, registry allowlisting, and automated review of all new dependencies introduced by AI-authored changes.
Where This Is Going
The current state of AI coding agents is early. The autonomy is partial, the oversight is often manual, and most organizations are still figuring out where agents fit in their workflow.
The trajectory is clear: the degree of autonomy will increase. Agents will handle more of the routine work — bug fixes, test coverage, dependency updates, documentation. The human’s role will shift from writing code to defining intent, reviewing decisions, and handling the novel problems that agents cannot pattern-match to prior solutions.
This is a good outcome — if the governance infrastructure is built alongside the autonomy, not as an afterthought. The organizations that treat this transition as primarily a productivity question will build AI pipelines that work well until they fail catastrophically. The organizations that treat it as a governance question from the start will build AI pipelines that can be extended safely as agent capability grows.
Your CI/CD pipeline is not a passive infrastructure component. When AI agents are in the development loop, it is the active enforcement point for everything your organization has decided about how software should be built, secured, and deployed. Build it accordingly.
The Bottom Line
AI agents in CI/CD pipelines are not a future scenario requiring future preparation. They are a present reality requiring present decisions.
The pipelines that were built for human developers are not wrong — they are incomplete. They need provenance tracking, AI-specific output validation, dependency integrity checks, and a human review process that is reoriented from mechanical correctness to intent alignment.
The good news is that the tooling exists. SLSA, Sigstore, OPA, Semgrep, GitOps operators — the components of an AI-aware pipeline are available today. The gap is not technology. It is the organizational decision to treat AI agent governance as a first-class engineering concern rather than a problem to solve after something goes wrong.
The pipeline is the last line of meaningful human oversight over AI-generated code. It deserves to be treated that way.
Is your organization already running AI agents in your development workflow? What governance gaps have you discovered? Share your experience in the comments.