Is There a Silver Bullet for Agents?

@Author: Garfield Zhu

Quote: I don't spend much time on Agent engineering and orchestration. Partly because I'm hilariously out of my depth—I've only read one file of Pi's source, the agent loop. Partly because LinkedIn and X feed me screenfuls of AI-generated Agent diagrams until I start feeling a little 🫠. I might rather spend that time seeing what new things people are building with agents on GitHub.

Today, while Codex and DeepSeek Harness were working on separate tasks, I took a break to read Addy Osmani's post about the Claude Code team's approach to code review. I expected to read about review throughput. I ended up thinking more about how much I can hand to an Agent—and where I still need to step in.

Code review catches bugs. It also tells people what happened.

The Claude Code team describes two jobs for code review: find and fix problems before a change ships, and help the team understand the codebase and the context behind a change. The second one doesn't always show up in an “Approve.” If someone can explain why a diff exists and which boundaries it touches, the next person spends less time trying to reconstruct the story from old code.

I came away thinking that review can reduce problems, but it can't drive them to zero. The requirement or acceptance criteria may have been off from the start; several reviewers can share the same blind spot; and a green test says only that the paths we exercised passed.

So I leave a few decisions to people: Are the goal and acceptance criteria clear? Is there enough evidence? Which residual risks can we live with, and how would we recover if something goes wrong? Agents can run plenty of checks, but people still have to decide what those checks are meant to establish. Within an agreed scope, Agents can inspect code, add tests, and watch CI. If the goal is wrong or access and data boundaries are off, running more loops just proves the wrong thing faster. Small diffs, regression checks, useful logs, feature flags, and rollback steps can make problems easier to catch, contain, and fix. What helps depends on the system.

The Claude team wrote that one CLI pull request went through more than 50 rounds of review and fixes overnight. They later used git blame to check the code reviewers kept flagging and found that about 59% of it was already there in the first round. Several rounds had surfaced old problems. The team capped the review loop at three rounds; after that, they reconsidered the approach and focused on reviewing the new diff. I didn't take away “three rounds is always safe.” I took away that when a loop stops moving, it's time to ask whether the change was split the wrong way. The Claude team also stresses that catching real bugs matters more than simply cutting the number of review rounds.

They look at the human time spent reviewing alongside reverts and incidents. That feels closer to the engineering trade-off I care about: how much human time goes into each change, how long an escaped problem takes to find and fix, and what recovery costs. We can count model calls too. I care more about whether the time saved actually goes back to the people doing the work.

Katelyn Lesse's post about multi-agent engineering made me think about the infrastructure that has to grow around these systems as we hand them harder work. That's useful engineering: it can expand what we can safely delegate. I just don't treat “humans never need to step in” as the finish line. A more useful question is whether people can get more done and spend less time correcting and cleaning up after the system.

A few situations I've run into on my projects

I've run into API changes where the contract draws hard lines around permissions, data scope, and callers. For changes like that, I'd have an Agent find the callers, gather the existing tests, and list the paths that might be affected. I still need to decide whether the request actually calls for changing permissions or data scope. Some repos also require negative tests and explicit human review for security-sensitive changes; that's a project rule, not something I'd assume everywhere.

Another example is this article itself. The build passed, but in the desktop preview I saw the status words in the table wrap onto two lines. I asked the Agent to adjust the column width and keep those labels on one line, then came back to check the page. A build tells me the content compiles; whether the table is easy to scan is something I have to see on the real page. It was a small call, but stepping in doesn't always mean writing code. Sometimes it's enough to say, “This looks off.”

How I use Agents

This is a working practice I've pieced together from situations in my own projects. I start with impact, permissions and data sensitivity, and how easy recovery would be. If a task fits more than one level, I use the higher one; if I can't tell, I move it up a level:

• R0 Documentation: spelling, explanation, article, or comments; no behavior change.
• R1 Local: a small change in one module, easy to reverse, with no external contract change.
• R2 Regular feature: ordinary user behavior or business logic with a bounded impact.
• R3 Cross-boundary: multiple modules, API or data contracts, migrations, bulk operations, or key configuration.
• R4 High risk: authentication or authorization, sensitive data, irreversible actions, or production release pipelines.

I've put the quick-reference matrix in the appendix below. If you don't feel like reading me ramble, you can just look up a level there.

The diagram is wide. On a phone, swipe sideways or open it at full size.

And yes, these are the diagrams that fill my LinkedIn and X feeds every day. But when I need to explain a workflow, I still ask AI to draw one. (If you can't beat 'em, join 'em.)

In the diagram, I've put the human calls at a few specific points: setting the goal and acceptance criteria, drawing access and data boundaries, sorting out a material disagreement between reviewers, and deciding who accepts production risk. Once the scope is clear, Agents can keep inspecting code, gathering context, running configured checks, and collecting CI results. We don't need to interrupt every routine step with another “Approve.”

How I dispatch subagents

• For multi-repo or multi-module exploration, I send read-only explorers to gather evidence in parallel. A spec reviewer checks for gaps in the request and acceptance criteria before implementation.
• Once the interface and file boundaries are clear, I can dispatch workers in parallel and give each one clear ownership. A worktree isolates files; it doesn't resolve a semantic conflict in a shared interface or at merge time.
• After implementation is integrated, a code reviewer checks the final diff. If the change touches permissions, sensitive data, dependencies, or the release path, I add a separate security reviewer. Both reviewers should compare the diff with the request, acceptance criteria, repo rules, and actual verification evidence.
• After a fix, review the new diff first. If it affects an earlier finding, widen the review to cover that conclusion too. I usually cap my review/fix loop at three rounds. If it still hasn't converged, I revisit the approach before we keep circling the same unresolved issue.

If the request or acceptance criteria change, a boundary around permissions, sensitive data, or irreversible actions is unclear, reviewers disagree on a material risk, or a production release is involved, I stop and make the call—or wait for the person the project says must approve it. Agents can run routine checks within the scope we've already agreed on. But if a required check can't be done, I can't report it as passed. This boundary is more useful than “a person must look at every step,” and more realistic than “the Agent should just do everything.”

I wouldn't measure this practice only by how many approval clicks it removes. I'd rather know how much human correction each accepted change takes, how long escaped problems take to surface, and how long recovery takes. If people step in less often but spend longer debugging and fixing what got through, we just moved the bill to another column.

Appendix: workflow configuration and risk matrix

I keep the project rules in AGENTS.md, the stages in a workflow, reviewer prompts on their own, and acceptance criteria separate from the actual commands. Here's the directory layout I have in mind:

project/
├── AGENTS.md
├── .agents/
│   ├── workflows/development.md
│   ├── reviewers/
│   │   ├── spec-reviewer.md
│   │   ├── code-reviewer.md
│   │   └── security-reviewer.md
│   └── verification/
│       ├── acceptance.yaml
│       ├── commands.json
│       └── run-record-template.md
├── scripts/
│   ├── verify.sh
│   ├── smoke-test.sh
│   └── regression-test.sh
└── .github/
    ├── CODEOWNERS.example
    └── workflows/validation.yml

# acceptance.yaml example for Agents and reviewers; this is not runner syntax
status_rules:
  conditional: "If the trigger applies, the check is required. Otherwise record not_applicable and explain why."
  unavailable: "If a required check applies but its command or environment is missing, record blocked/unverified."
  no_project_capability: "Record the capability gap and try an equivalent check; keep acceptance blocked/unverified if it is still unverified."
acceptance_evidence:
  record_in: .agents/verification/run-record-template.md
  include: [request_and_acceptance, triggered_checks, command_or_browser_evidence, uncovered_items]
review_policy:
  max_rounds: 3
  after_limit: "Revisit the approach instead of repeating the same review/fix loop."
  recheck_old_findings_if_affected: true
human_gates:
  require_decision: [scope_or_acceptance_change, auth_or_sensitive_data, destructive_change, conflicting_material_review, residual_risk_acceptance, production_release]
  agent_review_counts_as_human_approval: false
risk_levels:
  R3:
    required: [approved_spec_and_plan, spec_review, unit_tests, integration_or_contract_tests, regression_tests, full_relevant_ci, code_review, security_or_data_review, rollback_plan]
    conditional: [negative_tests, user_path_validation, e2e, preview_deploy, deploy, staged_rollout, post_deploy_validation, monitoring]
  R4:
    required: [human_scope_decision, spec_review, independent_code_review, security_review, negative_tests, regression_tests, full_relevant_ci, explicit_human_approval, rollback_plan]
    conditional: [production_deploy, staged_rollout_or_safe_equivalent, post_deploy_validation, monitoring]
stages:
  post_deploy: "Any actual deployment requires health and changed-path validation."
  staged_rollout: "Use a canary or flag when risk warrants it; R4 production release needs a safe staged rollout."
  rollback: "Name an owner, stop trigger, and executable recovery path before a risky release."
stop_conditions: [required_check_failed, acceptance_or_scope_changed, material_review_conflict_unresolved, rollout_signal_exceeds_project_threshold]
recovery: [owner, feature_flag_or_rollback_action, health_and_changed_path_recheck]

{
  "enabled": false,
  "profiles": { "R1": ["diff-check", "unit", "build"] },
  "commands": {
    "diff-check": ["git", "diff", "--check"],
    "unit": ["node", "--test", "scripts/i18n.test.mjs", "scripts/blog-i18n.test.mjs"],
    "build": ["yarn", "build"]
  }
}

This JSON is a small example, not a runner already wired into the blog. The unit and build commands are real blog-repo commands; the separate workflow-kit prototype has verify.sh call verify.py and run only the commands configured in the selected profile. It does not interpret acceptance.yaml, decide which conditions apply, or mark review and human approval complete. The example leaves enabled set to false; before enabling a profile in any project, add that repo's real lint and type-check commands too. If a required check applies but has no usable command or environment, keep it blocked/unverified. Use N/A only when a condition does not apply or the project genuinely lacks that capability and a reasonable alternative was used. Record stop conditions, an owner, and recovery steps; the runner will not roll anything back for you.

In the matrix, Optional means the risk level doesn't require the step by default. If a trigger in the details column applies, the check becomes required. Required applies to work the step actually covers. If the trigger doesn't apply, record N/A with a reason; if a required check can't run, record blocked/unverified. Any actual deployment requires post-deployment validation.

On a phone, swipe sideways to see the risk matrix. The headers scroll with the table, and the step labels won’t cover the other columns.

The matrix is a quick reference. For each task, record the triggered checks and the evidence in the run record.

What I want to do with the time I get back

I don't want to start measuring AI productivity by how many jobs disappear. Fewer people making the same amount will make output per person look better; if total output hasn't grown, I have a hard time calling that a productivity gain. It feels more like a short-term cost-cutting move before a bubble bursts. I'd rather see the same number of people—or even more—use AI to take on work we used to put off as too expensive, too slow, too tedious, or too low-priority, and produce several times as much, maybe dozens of times as much. There should be somewhere else for the saved time and attention to go.

The Industrial Revolution makes me think of more than machines taking over physical tasks. Machines expanded what people could produce, but they didn't decide what to make with that capacity or how to share the gains. AI feels similar to me: whether the extra capacity helps us take on more work or simply cuts jobs isn't a decision the tool can make.

Agent engineering is worth doing: it expands what a team can handle and takes repetitive coding, checks, and information gathering off our plates, freeing our limited attention for bigger, harder work. Robots take on more of our physical work; AI can take some of the repetitive mental work. But the tools can't choose whether that extra capacity produces several times as much or just leaves fewer jobs. Poor, glorious carbon-based creatures still have to pick the direction.