Behavior localization: the hidden bottleneck in every agent harness
New research from Tencent Hunyuan shows finding the code that needs to change is harder than writing the change. A 3-level behavior handbook improves plan quality 10-19pp while using fewer tokens.
TL;DR: The Harness Handbook paper (Tencent Hunyuan, arXiv 2607.13285) proves that finding the code that needs to change is harder than writing the change itself. A behavior-centric 3-level handbook improves localization by 10-19 percentage points while using 8-13% fewer planner tokens. A DeepSeek-V4-Pro planner with a handbook matches Opus 4.8 and GPT-5.5 on implementation-site localization. The principle generalizes: organize around what the code does, not where it lives.
Key takeaways:
- Behavior localization is finding every code site that implements a target behavior. It is the central bottleneck in harness evolution. Not model capability. Not context window size
- The Harness Handbook replaces file/symbol organization with a 3-level hierarchy: system overview (L1), component overview (L2), source-grounded unit details (L3)
- A DeepSeek-V4-Pro planner with a handbook matches Opus 4.8 and GPT-5.5 on localization. File-level F1 reaches 89.3% against the GPT-5.5 reference
- The largest gains appear for Search-Hostile requests (+33.3 percentage points) and Cross-file changes (+20 points). These are the cases where grep and code search fail
- Progressive disclosure is the mechanism. Reveal context in layers. Verify every candidate site against the live repository. Resync after every edit
How do production harnesses hide their own code?
I have been writing about agent harnesses for over a year. The 15-jobs framework. The 98-percent-harness insight. Code as Agent Harness. This paper is from Tencent HY LLM Frontier, Indiana University, University of Maryland, and University of Georgia, published on arXiv on July 14, 2026 (arxiv.org/abs/2607.13285). The project page is at ruhan-wang.github.io/Harness-Handbook/.
Every post makes the same point: the harness matters more than the model.
But there is a question I could not answer until this paper.
A few weeks ago, I needed to change how my agent harness handles tool retry. I knew the retry logic existed. I wrote it. But finding every site that participates in retry took longer than writing the change itself. The planner that decides to retry. The executor that runs the tool. The state manager that tracks retry counts. The config that sets the policy. Four files across three execution stages, connected through shared state, not through any import chain.
This is the problem the paper identifies: behavior localization. Every modification request describes what the system should do. But repositories are organized by files, functions, and modules. Those are different axes. The gap between them is where planning fails.
Why is finding code harder than changing it?
The paper tests two open-source agent harnesses: Terminus-2 (a Python terminal agent, 6 source files, 103 functions) and Codex (a Rust monorepo, 2,267 source files, 34,363 functions). For each, they created 30 realistic modification requests classified into three types:
| Type | What it simulates |
|---|---|
| Query (Q) | Modify existing behavior — a trigger, a timing rule, a control-flow decision |
| Cross-file (CF) | Add an end-to-end capability spanning files, schemas, and interfaces |
| Search-Hostile (SH) | Find implementations in mirrored copies, fallback branches, or cold paths |
The requests are natural-language descriptions with no hint of where the code lives. Exactly what a developer or coding agent receives in practice.
A DeepSeek-V4-Pro planner with read-only tools produces an edit plan for each request. The evaluation compares two arms: Baseline explores the repository directly. Handbook-Assisted uses the handbook to localize before planning. Three independent judges (GPT-5.5, Opus 4.8, and DeepSeek-V4-Pro itself) score every plan on localization accuracy, scope control, and reasoning quality.
The result: the handbook arm wins across every dimension.
How does a behavior handbook work?
The handbook replaces the file-symbol organization with a behavior-centric one. Three levels:
Level 1. System Overview. What execution stages exist. What the main loop does. How data flows between stages. A map of the entire harness.
Level 2. Component Overview. For each stage: its inputs and outputs, the state it touches, which functions live here, what it depends on from other stages. A legend for the map.
Level 3. Unit Deep Dive. Source-grounded entries linked to specific functions or files. GPS coordinates for every implementation site.
There is also a state-register view: which state variables are written and read across which stages. This catches cross-cutting dependencies that a file-based view hides.
The construction pipeline runs fully automatically:
- Static fact extraction. Parse the repository with language-specific parsers. Extract every function, its signature, its call edges, its source location. Deterministic — no LLM calls.
- Behavioral organization. Map each function to an execution stage. Two modes: function-as-leaf (when a trusted stage skeleton exists) or file-as-leaf (infer stages from file descriptions). Both use iterative LLM-assisted proposal and review.
- Hierarchical synthesis. Convert the stage-function mapping into the L1-L3 document tree. Every L3 entry is linked to a validated source location — the pipeline opens the repository and checks that the function is still there before including it.
The catch: this works because agent harnesses have execution stages. A React frontend or a Go backend does not. The handbook is designed for a specific class of systems that happen to be the most important ones right now — production coding agents.
What happens when a planner uses the handbook?
The modification workflow uses a process called Behavior-Guided Progressive Disclosure (BGPD):
- Stage selection. L1 identifies which execution stages are relevant. If the request is about tool retries, stages = Tool Execution + Planning + State Management.
- Register expansion. The state-register view adds stages that share state with the selected stages. If Planning writes to a value that Tool Execution reads, both get included.
- Entry selection. Within those stages, pick the most relevant L3 entries.
- Call-relation expansion. Follow call edges to find sites not directly in the selected stages but called from them.
- Source verification. Open the live repository. Resolve every candidate location. Keep only the sites that still exist and are still relevant.
The result is a verified evidence set: file paths, function anchors, and current source excerpts. The planner works from this, not from the full repository.
After the edit, the handbook auto-resynchronizes. It reparses changed files, updates affected entries, and freezes any locators that cannot be revalidated. The next request starts from the updated handbook.
How much does a handbook improve plan quality?
The paper’s results are consistent across every dimension measured.
| Metric | Baseline | Handbook | Gain |
|---|---|---|---|
| Plan quality win rate (Codex) | 28.3% | 38.3% | +10.0 pp |
| Plan quality win rate (Terminus-2) | 26.7% | 45.6% | +18.9 pp |
| Planner tokens used (Codex) | 102K | 89K | -12.7% |
| Planner tokens used (Terminus-2) | 58K | 53K | -8.6% |
| Wrong rate — zero overlap with reference (Codex symbol-level) | 44.4% | 18.5% | -25.9 pp |
| File-level F1 vs GPT-5.5 (Terminus-2) | 76.5% | 89.3% | +12.8 |
The important number is the second row. Handbook guidance improves plan quality while using fewer tokens. The improvement is not explained by a larger context budget.
Stratified by request type, the handbook effect is larger where you would expect it:
| Request type | Baseline win rate | Handbook win rate | Gain |
|---|---|---|---|
| Search-Hostile (Terminus-2) | — | — | +33.3 pp |
| Cross-file (Terminus-2) | — | — | +20.0 pp |
| Query (Codex) | — | — | +26.7 pp |
Search-Hostile requests improve by 33 percentage points. These are the requests that hide implementations in mirrored paths, fallback branches, and cold code. The difference is between “I cannot find it” and “I know exactly where to edit.”
How does a weaker planner match frontier models with a handbook?
This is the finding that made me re-read the paper.
All 24 comparisons across both harnesses, both reference models (Opus 4.8 and GPT-5.5), and both granularities (file and symbol) are higher for the handbook-assisted arm. F1 gains range from 5.0 to 18.8 points.
On Terminus-2, a DeepSeek-V4-Pro planner with a handbook reaches 89.3% file-level F1 and 93.3% precision against the GPT-5.5 reference. A weaker model with a behavior index matches the localization ability of a frontier model working from scratch.
Why it works: the handbook does not make the planner smarter. It makes the planner’s limited context go further. Instead of reading files to discover what exists, the planner reads the handbook to find what is relevant. The difference between “explore and hope” and “look up and confirm” is the entire gap between 28% and 39% win rates.
The wrong rate drops by 25.9 percentage points on Codex symbol-level and 13.3 points on Terminus-2. Zero overlap with the reference plan. The handbook does not only improve precision. It eliminates complete misses.
When does behavior localization not apply?
If your harness has a single file, a simple loop, or deterministic routing, you do not need a handbook. Grep works. The problem emerges at scale: hundreds of files, thousands of functions, execution stages that share state across module boundaries.
If you use a managed harness (AWS AgentCore, Claude Agent SDK) where the orchestration logic is abstracted behind API calls, the behavior-to-code mapping exists inside the service, not your repository. The harness is not something you modify.
The handbook is for teams that own their harness code. Teams that modify execution stages, add tool integrations, change state management, and need to find all the affected sites before they ship a change.
What should you do if you build agent harnesses?
The paper validates a direction I suspected but could not prove: the bottleneck in agent development is shifting from model capability to context quality. A coding agent with a good index of its own codebase outperforms one without, even when the second uses a 10x more capable model.
Three concrete things you can do today:
Document your execution stages. If you cannot list the stages your harness cycles through, you cannot map behavior to code. Start with a plain-text document: initialization, planning, tool execution, observation, termination. What state does each stage read and write?
Build a state-register view. Track every state variable across your harness. Which stage writes it? Which stages read it? Cross-stage state dependencies are the most commonly missed sites during modification.
Progressive disclosure. When your coding agent needs to modify the harness, do not dump 30 files into context. Start with the relevant stages. Let the agent pick one. Then reveal the source. The same principle that makes the Harness Handbook work applies whether you have a formal handbook or a rough architecture document.
FAQ
What is behavior localization? It is the task of finding every code location that implements a target behavior in a production agent harness. A modification request says what the system should do, but does not say where that behavior lives in the codebase.
How does the Harness Handbook work? It organizes implementation knowledge around runtime behavior, not file locations. Three levels: L1 describes the overall architecture and execution stages, L2 covers each stage’s responsibilities and dependencies, L3 links source-grounded implementation entries to the live repository.
Does the handbook help coding agents or only humans? Both. The paper’s evaluation uses a coding agent as the planner. A DeepSeek-V4-Pro planner with a handbook matches Opus 4.8 and GPT-5.5 on localization. The handbook reduces planner token use by 9-13% while improving plan quality by 10-19 percentage points.
Can I use this approach for any codebase? The handbook is designed for agent harnesses with execution stages, state registers, and a main loop. The principle (organize around behavior instead of files) transfers to any codebase. The specific L1-L3 hierarchy works best when the code follows a well-defined lifecycle.
What is BGPD? Behavior-Guided Progressive Disclosure reveals context in layers. First identify relevant stages, then select component-level entries, then drill into source-grounded details. Every candidate site is verified against the live repository before it enters the edit plan.
Related Posts
- Code as Agent Harness: what a 102-page survey tells us. The broader landscape of agent harness design and why it matters more than model selection.
- Your agent is 1.6% model. The rest is the harness.. What the Claude Code architecture looks like under the hood and why the harness determines reliability.
- AWS just turned the agent harness into a managed service. The managed harness approach and how it validates the 15-jobs decomposition model.
- Loop engineering: the production agent loop nobody talks about. Execution stages in practice and why most implementations get the loop wrong.
This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach\u00a0me\u00a0at\[email protected]
