Your CS degree won't save you. Your domain expertise will.
Anthropic's 400K-session study proves managers succeed at coding tasks as often as engineers. The hard truth: domain expertise matters more than your CS degree.
TL;DR: The most important finding from Anthropic’s 400K-session study isn’t the division of labor or the task value growth. It’s this: domain expertise matters more than coding ability. Managers who don’t write code succeed at coding tasks slightly more often than software engineers. The gap between novice and intermediate users is bigger than the gap between intermediate and expert. Coding agents are making a coding background less relevant to successful programming. But they’re making domain expertise more valuable than ever.
Key takeaways:
- Management occupations hit the highest verified success rate on coding tasks. Slightly above software engineers. The skill of specifying what you want transfers directly to directing an agent.
- The expertise gradient is front-loaded. Most of the gain comes between novice and intermediate, not intermediate to expert. Competence, not mastery, is enough.
- Novices abandon sessions at 3x the rate of everyone else (19% vs 5-7%). The value of expertise is partly about recovery: knowing how to steer when things go wrong.
- This is substitution at the execution layer, not the problem-solving layer. Agents replace the how. They don’t replace the what or the why.
- The right mental model: you aren’t an engineer working faster. You’re a domain expert who can now execute technical work directly.
A manager who doesn’t write code succeeded at coding tasks more often than a senior engineer with 10 years of experience. Not in a benchmark. Not in a lab. In 400,000 real sessions of people doing real work.
Anthropic’s Agentic coding and persistent returns to expertise paper published this week is being read as a data dump about Claude Code usage. It isn’t. It’s a map of how knowledge work is reorganizing around agents. And the map shows something uncomfortable if you’ve built your identity around being a programmer. The paddo.dev analysis of the “10x AI developer” myth makes a related point: vendor-funded productivity studies claim 55% faster completion, but independent replication doesn’t always hold.
Coding ability is becoming table stakes. Domain expertise is the differentiator.
What does the data actually say?
The occupation classifier in the paper is brutally honest. It doesn’t look at job titles. It reads session transcripts and infers what kind of professional you are from the context: the projects you reference, the vocabulary you use, the artifacts you care about. A lawyer writing a contract-parsing script gets classified as Legal. A product manager building a dashboard query tool gets classified as Management. Only sessions with clear signals of software or data work as the person’s job get classified into Computer and Mathematical Occupations.
The results challenge the “10x engineer with AI” narrative that dominates tech discourse.
In sessions that produce code, software engineers hit verified success 34% of the time. Management occupations: slightly higher. Business and financial operations professionals: around 33%. Life and physical scientists: around 32%. Arts, design, and media professionals: around 31%.
Every major occupation lands within 7 percentage points of software engineers.
Under partial success (the measure that captures whether the person got what they needed even if it wasn’t perfect), the gap nearly disappears. Software occupations: 89%. Everything else: 88%.
The paper’s authors are careful not to overstate this. They note that managers may be more likely to explicitly confirm when they get what they asked for, which could inflate the verified success signal slightly. But a systematic bias across all non-software occupations would need to be large and consistent to erase a 7-point gap entirely. The simpler explanation is the one the data supports: domain expertise transfers across occupational boundaries more effectively than coding ability.
Why does the expertise gradient matter more than the occupation finding?
The occupation finding is the headline grabber. The expertise gradient is the deeper signal.
The paper rates each session’s user expertise on a five-point scale based on three signals: precision of direction, what they ask the agent to verify, and who corrects whom. A senior engineer asking their first Rust question is a novice at that task. An accountant who has never used Python but specifies exact reconciliation rules is an expert.
| Expertise | Verified success | Partial success | When trouble hits, abandoned |
|---|---|---|---|
| Novice | 15% | 77% | 19% |
| Intermediate | 28% | 91% | 5-7% |
| Expert | 33% | 92% | 5-7% |
Three things stand out.
First, the biggest gain is between novice and intermediate. You don’t need deep mastery to get most of the benefit. You need a working understanding of the problem and the ability to communicate what done looks like. That’s accessible to more people than “expert” implies.
Second, the abandon rate tells a different story than the success rate. Novices abandon sessions at 3x the rate of everyone else. When things go wrong (an error, a failed test, unexpected output), they don’t know how to steer back. The paper finds that experts hit trouble less often overall, and when they do, the trouble is on harder problems (the estimated value of a troubled expert session is roughly double that of a troubled novice session). Part of expertise is simply knowing how to recover.
Third, the gradient persists even when controlling for work mode, task value, month, subject, and occupation. It isn’t that experts pick easier tasks. They pick harder tasks and still succeed more often.
Is the 10x AI engineer the wrong frame?
The dominant narrative around AI coding tools is that they make good developers great. A 10x engineer becomes 20x. A senior engineer becomes unstoppable.
The Anthropic data suggests this framing misses the main event. The real transformation isn’t that engineers get faster at engineering. It’s that domain experts can now execute technical work directly.
A supply chain manager who understands inventory optimization doesn’t need to learn Python and API design to build a demand forecasting tool. They can describe the business logic, the data sources, and the validation criteria, and Claude Code builds it. The engineering work gets absorbed. The domain expertise gets amplified.
This is fundamentally different from the “10x developer” narrative in two ways.
First, it expands the pool of people who can produce software, not just the output of people who already do. The share of Claude Code sessions from non-software occupations is growing. Management, sales, and legal are the fastest-growing groups in the data.
Second, it changes what productivity means for people who already code. The paper tracked the share of sessions spent debugging fall from 33% to 19% over 7 months. Implementation-heavy work is compressing. The remaining work is more about architecture, verification, and problem decomposition. The things that require domain understanding.
The kitchen analogy works here. A head chef at a busy restaurant doesn’t cook every dish. They know what every station is doing, when the risotto should be fired, and whether the pass is clear for the next ticket. The line cooks handle the execution. Coding agents are line cooks. The question is whether you know what to tell them and how to check their work.
Who is this not for?
If you’re early in your career and building coding skills, this doesn’t mean your effort is wasted. The paper’s findings apply to people who already have domain expertise in something (finance, biology, operations, management) and use Claude Code to execute technical work within that domain. If you’re learning to code as your primary skill, you’re building both coding ability and future domain expertise in how software works. That combination remains powerful.
The data also doesn’t capture the ceiling. It covers what people are doing with today’s Claude Code, not what’s possible. As models get better, the expertise gradient could flatten. Or it could steepen if more capable models amplify the advantage of better direction. The paper explicitly flags this as an open question.
And the success rates themselves are a wake-up call. Even expert-rated sessions only reach verified success 33% of the time. The tools are powerful but not automatic. Domain expertise improves outcomes dramatically, but it doesn’t guarantee them.
What should you do with this?
Three things.
First, invest in problem knowledge, not just tool knowledge. The developers who’ll be most valuable with AI coding tools aren’t the ones who know the most model names or prompting tricks. They’re the ones who understand the domain deeply enough to decompose problems and verify solutions. Every hour you spend understanding a business domain is an hour that compounds when you direct an agent to work in it.
Second, learn the skill of agent direction. The paper’s expertise classifier is instructive: precise direction, verification of output, and the ability to correct when things go wrong. These are teachable skills. They aren’t the same as coding. They’re closer to technical management: specifying what done looks like and inspecting what comes back.
Third, pay attention to the recovery rate. Novices abandon 19% of sessions when things go wrong. Experts abandon 5-7%. The difference isn’t that experts make fewer mistakes. It’s that they know how to recover. Building a personal library of recovery patterns (error messages you’ve seen, debugging workflows that work, validation scripts you trust) is a durable asset.
FAQ
Does this mean coding is dead? No. It means the value of coding is shifting from implementation to direction. The syntax matters less than the specification. The code matters less than the architecture.
Should I stop learning to code? If you’re early career, no. Learning to code builds mental models of how software works. That understanding transfers directly to directing agents. The risk is stopping at syntax and never developing domain expertise in any problem space.
Why do managers succeed more than engineers? The paper offers two explanations: managers may be better at specifying what they want and confirming when they get it, and they may communicate explicit affirmation more readily. Both point to the same underlying skill: the ability to direct work effectively.
Will the returns to expertise decrease as models improve? Possibly. The paper flags this as a key open question. If models start supplying the judgment that users currently bring, the advantage of domain expertise could narrow. But that’s a prediction about the future, not a description of what exists today.
What’s the one-sentence takeaway? Coding agents aren’t substituting for domain expertise. The more understanding a worker brings to an agent, the more quality work the agent is able to do.
Related Posts
- AI wrote the code. The hard part is everything else.. What comes after the code is written: testing, deployment, maintenance, and why that’s where the real work lives.
- Spec-Driven Development and the Vertical Agent Method. A framework for specifying what you want before you direct an agent to build it.
- The Vertical Agent Method: ship AI agents in 14 days. The named framework behind building production agents fast.
- How AI coding agents actually use your SDK. Understanding the interface between developers and agents.
This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected]
