Agentic DevOps: Speed Needs a Delivery System

AI coding tools can generate code faster than any team could type it by hand. That fact gets treated as the whole story a lot more often than it should. On this week's Code To Cloud podcast, I sat down with Paul Duvall — a longtime software engineering and DevOps practitioner who's spent his career on exactly this problem — to talk about what actually happens when you point a fast code generator at a slow delivery pipeline. Short version: the pipeline wins, and you get more work in progress, not more value delivered.

Listen to the Full Episode

Agentic DevOps Isn't a Replacement for CI/CD — It's What Makes CI/CD Worth Having

Agentic DevOps is the use of AI agents and coding tools inside a software delivery workflow, backed by the engineering practices that let a team build, test, deploy, and improve software reliably. That second half of the definition is the part that gets dropped. An agent can write code, run checks, and patch issues, but none of that reaches a customer without automated tests, a reliable deployment process, and fast feedback from both CI and production underneath it.

Paul's framing stuck with me: the goal was never "produce more code." It's reducing the time between an idea and a useful change reaching users. Those are different targets, and optimizing for the first one can actively work against the second.

Why Faster Code Generation Can Slow Delivery Down

More code out of an AI tool sounds like a clear win, right up until you notice that reviews, testing, and deployment are still running at the same speed they always did. When that happens, the queue doesn't disappear — it just moves. Pull requests pile up. Developers spend more time waiting on approvals and re-litigating feedback on changes that have been sitting for days. The team is visibly busier. Users don't see anything sooner.

That's the core of Paul's argument: a faster coding step doesn't make the whole value stream faster if a manual approval or an operational handoff further downstream still gates the release. His prescription is to stop measuring the coding step in isolation and ask about the system instead:

• How long does a change actually take to reach users?
• Where in the pipeline does work sit and wait?
• How early do tests and checks catch a problem?
• Do findings from production make it back into how the team builds the next change?

The value stream determines delivery speed — not how quickly the first draft of the code got written.

The CI/CD Fundamentals That Make an AI Coding Tool Worth Having

None of this is an argument against AI. It's an argument that the fundamentals matter more once AI is in the loop, not less. The practices that make a difference are the same ones that mattered before agents showed up:

• Automated tests that run with every change
• Static analysis and other repeatable checks
• Small changes on short-lived branches
• Fast feedback the moment a check fails
• Automated deployment where it's appropriate

These are what let a team trust automation — a change moves forward on evidence, not on a place in the same manual queue as everything else. Skip them, and an AI agent is just generating more raw material for a bottleneck that already existed.

Harness Engineering: Catch It Before It's a Pull Request

Paul calls the practical version of this "harness engineering" — building the checks and guidance around an AI coding tool so problems surface earlier in the workflow, not after a human reviewer has to find them. Concretely: define repository standards for the agent to follow, and run automated checks for things like secrets or unnecessarily complex code before a change ever opens a PR.

The underlying idea is making engineering expectations explicit and repeatable — document the practices, run the right checks close to where the code gets written, feed the agent useful signal when a check fails, and review each change according to its actual risk rather than treating every diff the same. It's the same instinct behind the harness work we walked through live at Prairie Dev Con — guardrails, approval gates, and scoped permissions are what separate an agent you'd trust near production from an impressive demo. This doesn't assume every AI-generated change is correct; it just moves the check to where a mistake is cheap to catch instead of expensive.

Close the Loop Between Production and Development

Code can pass every test and still behave differently once it's live — real data and real user behavior surface things a local check never will. That's why production feedback has to flow back into development. When a team finds a problem in production, fixing the immediate issue is only half the job; capturing the lesson in code, tests, or a delivery rule is what makes the fix durable instead of a one-time patch you'll make again in six months.

Progressive delivery — feature flags, canary releases, staged rollouts — is how you validate a change gradually instead of learning about a bad one from every user at once. The point isn't only shipping safely. It's learning from how the thing actually behaves and feeding that straight back into the next change.

Measure the Outcome, Not the Output

More generated code and more pull requests are not proof that customers are getting value faster — they're proof of activity. The measures that actually reflect delivery are the four DORA metrics: lead time for changes, deployment frequency, change failure rate, and time to restore service. They describe the flow and reliability of the whole process, which is the only place "is AI actually helping" can be answered honestly.

The better question isn't "how much more code can we generate." It's: can we validate changes sooner, release them safely, learn from users faster, and see clearly where work is waiting between an idea and production?

Where to Start

Paul's advice wasn't "automate everything at once." It was to build confidence in order:

1. Map the current workflow. Find out where changes actually wait — between development, review, testing, and release.
2. Strengthen CI first. Make sure the tests and checks that matter run consistently, every time, not just eventually.
3. Shrink the batch size. Short-lived branches and smaller changes make feedback easier to act on.
4. Write the standards down. Document what the team — and the agent — is expected to follow, and back it with automated checks.
5. Roll out progressively. Feature flags and staged rollouts let you observe a change before it reaches everyone.
6. Feed production back in. Turn recurring problems into permanent improvements to code, tests, or automation.
7. Measure the outcome. Track the four metrics above, not the volume of AI-generated commits.

The Takeaway

AI coding tools can absolutely help a team move faster. They just don't do it automatically, and they don't do it alone. Without reliable CI/CD, checks that run early, and a real feedback loop from production, faster code generation mostly manufactures more work waiting downstream. Agentic DevOps works when AI operates inside a delivery system built to catch problems early, validate consistently, and learn from what actually happens once software ships — which is the same system we build with clients under the agentic DevOps banner, just with the AI coding layer added on top of foundations that already have to hold.

If you want a quick read on which of those foundations your own team already has before you talk to anyone, the 5-minute agentic AI readiness scorecard is a faster starting point than a call.

Frequently Asked Questions

What is agentic DevOps?

Agentic DevOps is the use of AI agents and coding tools within a software delivery workflow, supported by the engineering practices that help teams build, test, deploy, and improve software reliably. It builds on DevOps and CI/CD rather than replacing them — an agent can write code, run checks, and fix issues, but the team still needs automated tests, reliable deployment, and fast feedback from CI and production for any of that to translate into value delivered sooner.

Does AI replace the need for CI/CD?

No. CI/CD is what validates and moves a change safely, and that need doesn't go away just because the code came from an agent. If anything it matters more — automated tests, static analysis, and fast feedback are what let a team trust an AI-generated change enough to act on it quickly instead of routing it into the same manual review queue as everything else.

Why can AI-generated code create a review bottleneck instead of speeding delivery up?

AI can raise the volume of changes a team produces faster than review, testing, or release can absorb them. If output rises while approvals and deployment stay manual, pull requests queue up and developers spend more time waiting and re-explaining feedback. The team looks busier, but users don't get value any sooner — which is why the right question is how long a change takes to reach production, not how much code got generated.

What is harness engineering?

Harness engineering, a term Paul Duvall uses for this, means building the checks and guidance around an AI coding tool so problems surface before a change ever reaches a pull request — documented repository standards the agent is expected to follow, and automated checks for things like secrets or overly complex code, run close to where the code is created. It shifts quality checks earlier without assuming every AI-generated change is automatically correct.

What should teams measure when adopting agentic DevOps?

The four DORA metrics — lead time for changes, deployment frequency, change failure rate, and time to restore service — because they describe the flow and reliability of the whole delivery process, not just how much code got produced. A rise in commits or pull requests proves activity; it doesn't prove customers are getting value faster.