The verification gap: output multiplied, checking did not
Your team ships more than ever and trusts it less than ever. That is the verification gap, and the teams furthest ahead on AI hit it first. This page covers how a team closes it period by period, and ends with the escape tracker and working agreement, free.
The people with the most AI named it first
Fiona Fung manages the Claude Code and Cowork teams at Anthropic, plausibly the most AI-leveraged engineering group anywhere. On Lenny’s Podcast in June 2026, she said her engineers ship about 8x more code than a year ago, and that the biggest problem this creates is verification: how do you know the experience you shipped is the one you intended? Her team’s answer includes tracking escapes as bad (unrecoverable, like a crash) versus sad (recoverable pain, like flicker), so quality has a shared vocabulary instead of a feeling.
The industry data says the same thing from the outside. LinearB’s 2026 benchmarks measured agentic-AI pull requests waiting over five times longer for first review than human-written ones. Generation scaled; the checking layer did not. Most teams respond by reviewing faster and trusting more, which is not a strategy, it is a slow leak. The gap shows up later as incidents nobody can explain and a codebase nobody fully knows, which is where this page meets its sibling, comprehension debt.
How Aurora Coach addresses it
Verification is not a tool you install; it is a set of habits a team agrees on and then actually keeps under delivery pressure. To be precise about the boundary: Aurora Coach does not run your tests or scan your code. It keeps the team’s checking habits from decaying when the sprint gets loud. Aurora Coach runs the loop that keeps them: Sense, Analyze, Recommend, Refine, Commit, Execute, and Re-evaluate. The verification gap lives mainly in the Engineering domain (testing automation, AI-assisted development), with Operations (recovery time, monitoring) alongside.
- Sense + Analyze Every team member contributes context: the AI asks structured questions, the team responds. Which merges people quietly doubt, where review attention actually goes; Aurora Coach for GitHub adds quantitative signal like diff sizes and review timing. The AI synthesizes it into a SWOT and maturity assessment grounded in the team’s actual situation plus industry best practice and research.
- Recommend + Refine + Commit The AI recommends concrete next steps with rationale, implementation steps, and success criteria: an evidence rule, a sampling rate, a diff-size budget. Team members vote, the team lead refines, the AI never decides. The team commits to what fits its reality.
- Execute + Re-evaluate The team does the work in its own context, alongside delivery. The next period’s analysis sees whether escapes moved: fewer bads reaching users, more catches happening earlier. Progress is visible, and the loop continues.
Free-text check-ins catch the early signal: the merge someone approved but did not believe in. For the review-process half of the problem, see code review for AI-generated code; when escapes become incidents, postmortem follow-through picks up.
What is the verification gap?
The distance between how much code a team ships and how much of it anyone has confirmed does what was intended. AI assistants multiplied output; testing, review, and QA capacity did not multiply with it. The gap is invisible while things work and expensive when they stop.
What is the bad versus sad framework?
A quality classification used by Anthropic’s Claude Code team, described by Fiona Fung, who manages the team. Bad is an unrecoverable failure such as a crash or silent wrong result; sad is a recoverable pain point such as flicker or slowness. Tracking both, with where each escape came from and why, shows a team where its verification actually leaks.
How do you verify AI-generated code without reading every line?
You stop pretending line-reading scales and verify at the level of intent and evidence: state what the change should do before generating it, attach proof it does that to every merge, deep-read a sample of agent diffs to keep comprehension alive, and log every escape so next period’s checks aim at real leaks instead of guessed ones.
The bad-vs-sad escape tracker, free
Log every bug that reaches a user with these six fields. Two weeks of entries tells you more about your verification than any dashboard.
- What the user hit One sentence describing the experience, not the code. "Export button spins forever", not "race condition in the queue worker".
- Bad or sad Bad is unrecoverable: a crash, data loss, a wrong answer returned silently. Sad is recoverable pain: flicker, slowness, a confusing state the user can back out of.
- Where it came from Human-written, agent-written, or mixed. Not to assign blame; to see whether escapes cluster by source.
- Who caught it A test, a monitor, a reviewer, a teammate, or a user in production. The further right on that list, the bigger the verification gap.
- Why it escaped The honest reason: no test covered it, the diff was too big to read, the reviewer trusted the green checkmark, nobody ran the flow.
- Smallest check that would have caught it One concrete verification step. This field is the whole point: it converts an escape into next period’s improvement.
The verification working agreement
The tracker shows where verification leaks. These five commitments plug the leaks:
- Intent written before generation A sentence on what the change should do, written before the agent runs. Verification needs a spec; "looks right" is not one.
- Evidence attached to every merge A test, a replayed flow, or a screenshot of the working behavior. The author proves it does what the intent said, the reviewer checks the proof.
- Deep-read sampling A fixed share of agent-written diffs gets a line-by-line human read each period. Sampling keeps comprehension alive without pretending you can read everything.
- Escapes get logged, bad or sad Every bug that reaches a user goes in the tracker with source and why-it-escaped filled in. No log, no learning.
- The tracker gets read Once a period, the team looks at where escapes cluster and picks one verification improvement. That closes the loop; the log alone changes nothing.
The tracker and agreement are yours to lift. A free trial keeps them alive when delivery pressure hits; MapROI (free, no signup, about 5 minutes) maps the ROI for your team's actual AI usage.