--- title: AI wrote the code. The hard part is everything else. canonical: "https://agenticup.dev/posts/ai-wrote-the-code-hard-part-everything-else/" pubDate: "2026-06-15T00:00:00.000Z" description: "Three bottlenecks AI can't touch: deciding what to build, verifying it's right, and the deep system understanding required for both. Evidence from the decide-execute-deliver sandwich model." tags: [ai-agents, software-engineering, productivity, agent-architecture, opinions] --- **TL;DR:** AI compresses the "execute" middle of the software engineering sandwich. It can't automate the two ends: deciding what to build and verifying what shipped. Evidence from a new essay by Arvind Narayanan and Sayash Kapoor shows that every high-profile claim of AI-driven layoffs falls apart under scrutiny. The NY WARN Act data is the cleanest confirmation: out of 25,000 laid-off workers, only 46 were affected by AI. **Key takeaways:** - The sandwich model: AI collapses code writing, but the deciding and verifying layers resist automation structurally, not because of capability gaps - Three bottlenecks: deciding what to build, verifying what ships, and the deep human understanding required for both - Every major "AI-driven layoff" story (Block, Snap, Intuit) was AI washing of financial cuts. Not a single company checked the AI box on NY WARN Act filings in the first year - 8x more LOC written by AI agents led to only 30% more releases. The bottleneck isn't code - If your value is in execution (writing code from specs), AI compresses your role. If your value is in judgment and accountability, AI makes you more valuable Two months ago, the contractor vanished. Not literally. He stopped answering messages. Three weeks into building a data pipeline, he disappeared with the API keys, a half-finished orchestrator, and the only copy of the schema documentation. I fired up Claude Code, fed it the requirements doc, and fifteen minutes later it had a working pipeline. SQLite ingestion, retry logic, error notifications. The whole thing. I leaned back. *This is it. I don't need people anymore.* The pipeline ran for exactly one day. Then the source API changed its response format. The agent's error handling caught the exception and retried. And retried. And retried. Eleven thousand failed requests later, the data was corrupted, the source banned the IP, and I had learned the first rule of shipping with AI. Writing code isn't the bottleneck. It never was. ## The sandwich model Arvind Narayanan and Sayash Kapoor, the authors of *AI Snake Oil*, published an essay this week that gives this intuition a rigorous framework. They call it the **decide-execute-deliver** sandwich. Every software project has three conceptual layers. Not sequential phases. You bounce between them constantly. But they are: - **Decide.** What to build. Problem framing, specification, planning. User needs, market signals, organizational priorities, regulatory constraints. - **Execute.** Design and implementation. The part AI is good at. - **Deliver.** Testing, verification, integration, maintenance. Accountability for what ships. AI compresses the middle layer. It can turn a specification into working code in minutes. But the other two layers resist automation. Not because the models aren't smart enough yet. Because they are structurally different kinds of work. ### The evidence A paper from earlier this year studied 100,000 developers on GitHub across generations of AI coding tools [Writing Code vs. Shipping Code](https://www.nber.org/papers/w35275). The title says it all. The numbers: - **Eight-fold increase** in lines of code written by developers using AI agents. - **Only 30% more releases.** AI collapsed the execution layer. But the decide and verify layers barely budged. Eight times more code produced. Less than a third more software shipped. The bottleneck shifted from writing code to every other part of the job. This isn't speculative. This is the data from 100,000 developers using the tools that exist today. ## Bottleneck one: Deciding what to build One of the hardest lessons junior engineers learn is that requirements take longer than code. Specifications aren't documentation. They are negotiation under uncertainty. The user doesn't know what they want until they see what they don't want. The market shifts mid-development. The org chart changes the priority. AI can generate a CRUD app from a prompt. It can't tell you whether the CRUD app is the right thing to build. That requires understanding the user, the business model, the competitive landscape, and the tradeoffs the team is willing to accept. Narayanan and Kapoor cite Fred Brooks 1986 essay *No Silver Bullet* here. Brooks distinguished between essential complexity (the inherent difficulty of specifying correct software behavior) and accidental complexity (limitations of the tools and languages). AI removes accidental complexity. It can't touch essential complexity. As AI capabilities improve, the kinds of decisions delegated to AI increase. But this doesn't make the decide layer thinner. It shifts the boundary. Once a decision can be delegated, it is no longer a source of competitive advantage, and human judgment migrates upward toward harder decisions. Software complexity grows to fill the gap. There is no ceiling. ## Bottleneck two: Verifying what ships The second bottleneck is accountability. Someone has to stand behind the code that ships. This isn't just about catching bugs. It's about being responsible for what the system does in production. An agent can write a function that handles user data. It can't certify that the function complies with GDPR, SOC 2, and the contract the legal team signed last quarter. It can't know what the company's liability exposure looks like for this specific feature. The SWE-chat dataset, which logs coding agent interactions from open-source developers who opted into a monitoring tool, found that only **44% of agent-produced code survives to user commits**. More than half gets discarded during review. And vibe-coded commits, where the developer doesn't review what the agent produced, introduce vulnerabilities at **nine times the rate** of human-only code. Even in the most optimistic scenario, verifying agent output is real work. Simon Willison, one of the most thoughtful chroniclers of the AI transition, said he is *mentally exhausted by 11am* from supervising agents. Supervision isn't passive monitoring. It's constant decision-making about whether the agent is doing the right thing. ## Bottleneck three: Deep human understanding The third bottleneck is the prerequisite for the first two. You can't decide what to build or verify what shipped unless you deeply understand the codebase, the business, and the environment. This understanding is tacit. It accumulates through context that doesn't fit into a prompt. Why was that function written in that specific way? Which deployment script has the workaround for the infrastructure quirk that nobody documented? What did the customer mean when they said "fast" in the meeting last week? AI agents operate on what is written down and in context. Humans operate on what is known but never written. That gap is the third bottleneck and the one that is hardest to automate because it isn't a capability gap. It's a knowledge representation gap. ## The data on AI washing The claim that AI is replacing software engineers should be easy to verify. It's happening right now, supposedly. There should be data. There is. And it tells the opposite story. **Block (Jack Dorsey).** February 2026. 4,000 layoffs blamed on AI "enabling a new way of working." Actual reason: the company grew headcount threefold during the pandemic and was under massive financial pressure. A data scientist on the Cash App team said Block "shoved AI down everyone's throats" and reported "very limited gains in productivity." She refused a 75% retention raise and quit. **Snap (Evan Spiegel).** April 2026. 1,000 layoffs blamed on AI. Spiegel said AI generated 65% of new code. Actual reason: activist investor demanded cost cuts. Snap has posted a net loss every full year since its 2017 IPO. The cuts hit the augmented reality division hardest. Not the kind of cuts you'd expect from AI-driven restructuring. **Intuit.** May 2026. 3,000 cuts, announced alongside deals with Anthropic and OpenAI. The press connected the dots. For once, the CEO pushed back. He said the cuts had "nothing to do with AI" and targeted "coordination-heavy roles" and too many management layers. These aren't cherry-picked exceptions. Every high-profile story of AI-driven layoffs that the authors examined in their [essay](https://www.normaltech.ai/p/why-ai-hasnt-replaced-software-engineers) had the same pattern: the AI explanation collapsed under scrutiny. **The NY WARN Act data is the cleanest confirmation.** In March 2025, New York became the first state to add an AI disclosure checkbox to mass layoff filings. In the full first year, more than 160 companies filed notices covering about 25,000 workers. **Not a single one checked the AI box.** As of late May 2026, the NY Department of Labor confirmed that only one company ever checked it: Nespresso. Forty-six workers out of 25,000. About two-tenths of a percent. The survey data is even more damning. A Forrester analyst put it bluntly: "When we ask if they have a mature, vetted AI app ready to fill in those jobs, nine out of 10 times, the answer is no and they haven't even started." A Harvard Business Review survey of over 1,000 global executives found that 21% had made large headcount reductions **in anticipation of** AI. Only 2% had made large reductions **because of actual AI implementation**. A 10x gap between expectation and reality. And 59% of US hiring managers admitted they emphasize AI when explaining layoffs because it "plays better with stakeholders" than citing financial constraints. ## What this means for engineers None of this means your job is safe. The authors say that explicitly in the next essay in their series. Aggregate demand for software engineers may be stable, but structural shifts will hit some engineers hard. The sandwich model tells you where the risk concentrates: - If your value is in the execution layer, writing code from a specification, AI will compress your role. This has been happening for decades. The Bureau of Labor Statistics started tracking "programmer" separately from "software engineer" twenty years ago. Programming has been shrinking and paying less ever since. - If your value is in deciding what to build, verifying what ships, and maintaining deep understanding of complex systems, AI makes you more valuable. It removes the grunt work from your day and raises the ceiling on what you can deliver. The future of software engineering looks like a crane operator, not a bricklayer. The agent does the heavy lifting. The human keeps it under control. That requires judgment, context, and accountability. The three things the sandwich model says won't be automated. Jevons paradox applies here. When something gets cheaper to produce, demand for it rises. Software has no demand ceiling. Modern cars run on 100 million lines of code. Hospitals need custom software for every department. Every industry that touches software will want more of it as the cost of production falls. Programmer employment in the US went from near-zero in 1950 to millions today. Agriculture, where automation destroyed jobs, had a fixed demand ceiling. People don't eat 25% more food because it got cheaper. Software has no such limit. The contractor who vanished? I never found him. But I also never hired a replacement. I still use agents to write the first pass. But now I spend the first hour of every day deciding exactly what the agent should build and the last hour verifying what it did. The middle part got fast. The ends took over my schedule. --- ## FAQ > **Won't AI eventually automate deciding what to build too?** > Some of it, yes. But as decisions get delegated to AI, the valuable human decisions migrate upward. Software complexity increases over time. There is no ceiling on what needs to be decided. > > **Is the decide-execute-deliver framework specific to software engineering?** > No. It applies broadly to knowledge work. The effect is furthest along in software because AI capabilities are strongest there. > > **Does vibe coding disprove the sandwich model?** > Vibe coding is a specific practice at one end of a spectrum. It works for throwaway scripts. It doesn't scale to production. The SWE-chat study found vibe-coded commits introduce vulnerabilities at nine times the rate of human-only code. > > **How does Simon Willison describe supervising AI agents?** > He said he's mentally exhausted by 11am from supervising agents. Supervision is constant decision-making about whether the agent is doing the right thing. ## Related Posts - [Your agent finished. That doesn't mean it worked.](/posts/agent-evaluation-task-completion-trajectory/). Task completion is a vanity metric. Trajectory-level evaluation catches what the agent missed. The verification half of the sandwich. - [AI agent state machines](/posts/ai-agent-state-machine/). The execution layer needs structure. State machines keep agents from running the same tool call 11 times. - [We're treating AI agents like magic tricks instead of software](/posts/agents-break-in-production/). Five predictable failure modes that hit exactly because the decide and deliver layers get skipped in the rush to ship. - [Proactive agent design patterns](/posts/proactive-agent-design-patterns/). When agents take initiative, the decide layer shifts from human to machine. Here's when that works and when it backfires. --- This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at hello@agenticup.dev