The verification gap: output multiplied, checking did not
Your team ships more than ever and trusts it less than ever. That is the verification gap, and the teams furthest ahead on AI hit it first. Free to copy on this page: the bad-vs-sad escape tracker and a five-point working agreement. It ends with how Aurora Coach turns escaped bugs into changes the team commits to, shown on a worked example.
The team with the most AI hit this first, and wrote down what worked
An agent writes a change in minutes. Confirming it does what you intended still takes a person, and that person has the same hours they had last year. Review turns into approval, and nobody answers whether what shipped is what you meant.
Anthropic’s Claude Code team ran into this at roughly 8x output, and their engineering manager Fiona Fung named verification the biggest problem it created. The tracker and the working agreement below are built on how her team handles it. The wider data says the same: LinearB’s 2026 benchmarks found agentic-AI pull requests waiting over five times longer for a first review.
Reviewing faster and trusting more is not a strategy. It is a slow leak.
The bad-vs-sad escape tracker, free
Log every bug that reaches a user with these six fields. The bad-or-sad split is borrowed from Fung’s team; the five fields around it are what turn a log into a change.
- What the user hit One sentence describing the experience, not the code. "Export button spins forever", not "race condition in the queue worker".
- Bad or sad Bad is unrecoverable: a crash, data loss, a wrong answer returned silently. Sad is recoverable pain: flicker, slowness, a confusing state the user can back out of.
- Where it came from Human-written, agent-written, or mixed. Not to assign blame; to see whether escapes cluster by source.
- Who caught it A test, a monitor, a reviewer, a teammate, or a user in production. The further right on that list, the bigger the verification gap.
- Why it escaped The honest reason: no test covered it, the diff was too big to read, the reviewer trusted the green checkmark, nobody ran the flow.
- Smallest check that would have caught it One concrete verification step. This field is the whole point: it converts an escape into next period’s improvement.
The verification working agreement
The tracker shows you where verification leaks. These five commitments are what stop the leaking, and none of them needs a tool or a budget line.
- Intent written before generation One sentence on what the change should do, written before the agent runs. Verification needs something to check against, and "looks right" is not it.
- Evidence attached to every merge A test, a replayed flow, or a screenshot of the working behavior. The author proves it does what the intent said, and the reviewer checks the proof instead of re-reading the whole diff. The rest of that shift is code review for AI-generated code.
- Deep-read sampling A fixed share of agent-written diffs gets a line-by-line human read each period. One in five is a workable start. The goal is not catching every bug, it is keeping someone able to explain the system, because what erodes without that is comprehension debt.
- Escapes get logged, bad or sad Every bug that reaches a user goes in the escape tracker with its source and why-it-escaped filled in. No log, no learning.
- The tracker gets read Once a period, the team looks at where escapes cluster and picks one verification improvement. The log on its own changes nothing. For the escapes that became incidents, the same discipline is postmortem follow-through.
What is the verification gap?
The distance between how much code a team ships and how much of it anyone has confirmed does what was intended. AI assistants multiplied output; testing, review, and QA capacity did not multiply with it. The gap is invisible while things work and expensive when they stop.
What is the bad versus sad framework?
A quality classification used by Anthropic’s Claude Code team, described by Fiona Fung, who manages the team. Bad is an unrecoverable failure such as a crash or silent wrong result; sad is a recoverable pain point such as flicker or slowness. Tracking both, with where each escape came from and why, shows a team where its verification actually leaks.
How do you verify AI-generated code without reading every line?
You stop pretending line-reading scales and verify at the level of intent and evidence: state what the change should do before generating it, attach proof it does that to every merge, deep-read a sample of agent diffs to keep comprehension alive, and log every escape so next period’s checks aim at real leaks instead of guessed ones.
From escaped bugs to changes: a worked example
Simulated team · Real product output Verification is a set of habits, and habits decay when the sprint gets loud. Aurora Coach works on the habits, not the scanning. Here is that on one team: Vantora Labs, a fictional Series B SaaS. Seven engineers on the core platform, and merged pull requests roughly tripled since the company funded coding agents for everyone. We scripted the inputs and ran them through Aurora Coach in production. Everything below is the product’s real output.
1Sense and analyze
Every team member answers structured questions in their own words, and can take any thread further with the coach in a check-in. Escaped bugs surface in the words of the people who merged them, before they are a metric.
The bug that passed every check
An engineer opens a check-in with a real escape: a June rounding bug in billing that reached paying customers. Tests green, review bot approving, and no human ever ran the flow. The coach names the failure mode as “agent-written tests validating agent-written logic”. He pushes back for something he can apply on Monday and gets a merge gate: “Did a human run this flow end-to-end in a test environment? If no, it doesn’t merge.” The person who submits the change owns that check, not the reviewer.
Nine merged, three verified
Another engineer, counting honestly: nine pull requests merged last week, about three verified beyond the tests the agent wrote for itself. He wants a rule, not a philosophy, and gets three triggers for a line-by-line read: shared infrastructure other teams depend on, patterns the agent has not used before, and complex business logic. Everything else gets a skim, because “you’re calibrating whether the agent’s judgment aligns with yours on stuff that matters.”
2Recommend, refine, commit
The period’s sessions roll up into a team analysis with category scores and recommendations, and those become concrete suggestions the team votes on and commits to. The AI informs the decision, it does not make it.
From escape to commitment
The working agreement above is general knowledge. This is what it becomes for one team. Read the Context field: it argues from their own situation, not a template. They deploy many times a day, their pipeline gates scored two of five, and their post-mortems already name the repeat offenders nobody turned into a check.
3Execute and re-evaluate
The team does the work inside normal delivery. The next period’s escape log is the exam: fewer unrecoverable bugs reaching users, and more of them caught before merge. Vantora has run one period. The trend view starts when the second one lands.
This is one use case. How the full product works is on the product overview.
Not ready to change anything today? You already have the tracker and working agreement above, copy buttons and all. If you want one improvement loop like these in your inbox each month, leave your email.
What this page cannot tell you is which of it applies to your team, this quarter. Aurora Coach works that out from your team's own words, recommends next steps with the reasoning, and the next period shows whether it held.
Both are free. The ROI mapper needs no signup and takes about two minutes. What team members write stays private to them: see AI governance.