7 min read AI Agents

The harness, not the model: what actually determines your AI agents' behavior

The harness, not the model: what actually determines your AI agents' behavior

A team moves from Top 30 to Top 5 on a reference benchmark. Without changing the model. Without a single extra dollar of compute. Only by reconfiguring what surrounds the model.

This is the documented experience on Terminal Bench 2.0 from teams that started working seriously on what the industry now calls Harness Engineering. Another team gained 13.7 points on the same benchmark by adjusting only the system prompt, available tools, and middleware, with an identical model. These numbers raise an uncomfortable question: if teams spend most of their time choosing and negotiating model contracts, are they working on the right problem?

Visual representation of the Agent = Model + Harness equation, with the harness accounting for 90% of behavior
The harness accounts for 90% of what determines an AI agent's behavior in production.

What teams consistently get wrong

When an agent makes a mistake, the first instinct is to blame the model. Wait for the next version. Switch to a competitor. That's understandable, but most of the time, it's wrong.

An analysis published by Google in May 2026 on the new agentic SDLC proposes a simple equation: Agent = Model + Harness. The model is the raw engine. The harness is the car, the road, and the highway code. A Formula 1 engine without a chassis, tires, or brakes doesn't finish the race.

The harness encompasses everything that frames the model: instruction and rule files (CLAUDE.md, AGENTS.md), tools and MCP servers the agent can access, sandbox execution environments, orchestration and delegation logic between sub-agents, deterministic guardrails and hooks, and full observability (logs, traces, cost, latency). By available measurements, the model accounts for roughly 10% of an agent's observable behavior in production. The harness, the remaining 90%.

The practical consequence: most agent failures are configuration failures, not model failures. A missing tool. A vague rule. An absent guardrail. A context saturated with noise. These are elements the team controls, not the vendor.

Agentic harness anatomy in concentric circles showing the model at center surrounded by 6 functional layers
Anatomy of the agentic harness: the model at the center, surrounded by the 6 layers that determine its actual behavior.

The Factory Model: the developer no longer produces code

To understand why the harness is central, you need to shift your mental model of what a developer produces in 2026.

In the traditional model, the developer writes code. In the agentic model, the developer designs the system that produces code. This is what the Google report calls the Factory Model: the analogy of a factory director who doesn't manufacture each part themselves, but designs the assembly line, defines quality controls, and ensures every output meets standards.

The "Developer Zone" covers: defining specs, designing guardrails, validating outputs. The "Agent Factory Floor" executes: a Planning Agent structures the work, a Coding Agent implements, a Tests & Verification system validates, and failures are sent back in an automatic feedback loop. The developer is no longer in the implementation loop; they are in the supervision loop.

This shift has a direct implication for valued skills. The question is no longer "how many lines of code do you write?" but "how precise are your specs, how exhaustive your tests, how well-calibrated your guardrails?" The most effective developers in teams that have adopted this model are those who can delegate with precision, not those who code the fastest.

The Factory Model with two zones: Developer Zone at top (specs, guardrails, validation) and Agent Factory Floor at bottom (planning, coding, tests, feedback loop)
The Factory Model: the developer designs the production system, not the parts themselves.

Conductor and Orchestrator: two modes, not two roles

In practice, the same developer alternates between two postures depending on the nature of the task. The Google report names them conductor and orchestrator.

The conductor works in real time, synchronously, in their IDE. They watch the code appear, guide the agent keystroke by keystroke, maintain granular control over every change. This is the natural mode for complex code, debugging on unfamiliar systems, or exploration. The tools that support this mode — GitHub Copilot, Cursor, Windsurf, Gemini Code Assist — are today the most widespread. The risk is identified: if the developer directs every keystroke, the AI's productivity gain is mechanically limited.

The orchestrator operates at a higher level of abstraction. They define an objective, assign it to one or several agents, and come back to review the outputs. Agents work in parallel, in the background, sometimes for hours. The developer validates, adjusts the trajectory, approves PRs. This mode suits well-defined tasks: fixing documented bugs, implementing features against established patterns, framework migrations, generating test suites. Tools in this category include Google Jules, GitHub Copilot agent mode, and agentic terminals like Claude Code or Codex CLI.

Most developers use both modes in the same day. It's not a tool choice, it's a posture choice depending on the task. A team's operational maturity is measured in part by its ability to explicitly distinguish when to use each.

The 80% problem: what benchmarks don't show

Productivity studies regularly announce 25 to 39% gains on development tasks. A METR study published in July 2025 brings an important nuance: experienced developers using AI assistants (Cursor Pro + Claude 3.5/3.7) took 19% longer on certain tasks, due to time spent verifying, debugging, and correcting outputs. A February 2026 update nuances this finding: with late-2025 tools, developers are likely faster, but selection bias (developers refused to work without AI, distorting the sample) made measurement unreliable. What both studies confirm: AI doesn't eliminate implementation work — it transforms it into review, guidance, and verification work.

This paradox is explained by what the same report calls the "80% problem." Agents quickly generate roughly 80% of a feature's code. The remaining 20% — edge cases, error handling, integration points, subtle correctness requirements — demands deep contextual knowledge that current models don't possess. And the nature of errors has evolved: you no longer see syntax mistakes. You see code that "looks correct," passes basic tests, but embeds false assumptions about business logic or creates invisible maintenance burdens.

The posture that works: use AI for what it does well (fast implementation of well-specified tasks) and reserve your attention for domains where it fails (ambiguous requirements, architectural trade-offs, correctness verification). Don't try to go faster by accepting everything the agent produces. Go faster by concentrating your expertise where it creates the most value.

Output evaluation vs trajectory evaluation: the distinction that changes everything in testing

Testing agent-generated code isn't limited to checking that tests pass. That's output evaluation, and it's necessary but insufficient.

You also need to evaluate the trajectory: the complete sequence of tool calls and intermediate reasoning the agent followed to reach the result. A smooth output that skipped its verification steps is more dangerous than an output that clearly failed — because the first error is invisible and arrives in production.

Teams that have implemented this dual evaluation define explicit rubrics: task success, quality of tool use, trajectory compliance, absence of hallucinations in dependencies, coverage of realistic failure modes. The rule set by Google for engineering leaders is clear: a working demo proves the agent can succeed once. An eval suite with rubrics demonstrates it succeeds reliably. The two are not interchangeable.

Comparison between output evaluation (final result) and trajectory evaluation (sequence of actions) for AI-agent-generated code
Dual evaluation of agentic code: output evaluation (final result) and trajectory evaluation (action sequence).

The real economics of the agentic SDLC

The conversation about AI in development often starts and ends with velocity. For a decision-maker, the relevant metric is total cost of ownership (TCO).

Vibe coding — using an agent with minimal structure, in prompt-and-accept mode — appears cheap: low upfront investment, fast early results. But this approach accumulates three compounding operational burdens: token burn rate (unstructured prompts create trial-and-error loops that burn tokens at low first-pass success rates), maintenance tax (code generated through ad hoc prompting lacks structural consistency and costs days to debug six months later), and security remediation (without an automated evaluation harness, fast code generation creates fast vulnerability generation, exponentially more costly to fix in production than at design time).

Agentic engineering inverts this model. The upfront investment is higher: designing API schemas, building deterministic test suites, structuring the agent's context. In return, the marginal cost per feature drops significantly. Teams that have crossed this inflection point report that vibe coding costs 3 to 10 times more per feature over time, compared to agentic engineering on a mature codebase.

One often under-exploited complementary lever: intelligent model routing. Using a large frontier model for high-complexity tasks (requirements, architecture, initial implementation) and automatically routing deterministic tasks (test generation, stylistic code review, CI monitoring) to smaller, less expensive models. A well-designed factory orchestrates a multi-model ecosystem rather than running everything through a single premium model.

What this means for engineering teams

Three practical implications emerge for teams looking to move from vibe coding to agentic engineering.

Treat context engineering as a full engineering discipline. Rule files, system prompts, eval suites, and skill libraries must be versioned with the project, reviewed in PRs, and owned by named engineers. Without this discipline, the harness drifts and agent behavior becomes irreproducible from one run to the next.

Explicitly distinguish prototyping work from production work in team norms. Vibe coding is valid for exploration and proofs of concept. Agentic engineering is mandatory for anything that ships. Leaving this boundary blurry produces prototypes that accidentally go to production.

Reframe recruiting and skill development around judgment, not implementation. As implementation becomes faster and more automated, the bottleneck shifts to specification, evaluation, architectural judgment, and review. The most valuable engineers in the coming years will be those who can direct agents with precision, not those who write the most code.

The closing formula from the Google report captures it: "Generation is solved. Verification, judgment, and direction are the new craft."

Sources

Godwin AVODAGBE