8 min read Agent Architecture

Maybe your agent should not be an agent: the five-step determinism test

Maybe your agent should not be an agent: the five-step determinism test

Gartner expects more than 40% of agentic AI projects to be cancelled by the end of 2027, for three reasons stated in its 25 June 2025 press release: escalating costs, unclear business value, inadequate risk controls. The firm adds a figure that describes the state of the market better than the forecast itself: out of the thousands of vendors presenting themselves as agentic, it estimates roughly 130 are legitimate, the rest falling under what it calls agent washing, the rebadging of existing assistants, RPA and chatbots. The forecast is still being picked up a year later, including by Forbes on 7 July 2026, which suggests the past year of deployments has not contradicted it.

The usual reading is that models are not good enough yet and the next generation will fix it. This year's measurements say something else. Run the same agent twice on the same task and it will overwhelmingly pick the same tools in the same order, but it will not call them with the same arguments, and 60% of that divergence appears in the first two steps of the chain. The problem is not loop length, tool count, or model size. It sits at the entrance. Where the request is ambiguous, an agent contributes something. Where it is not, the agent adds variance without adding capability.

The 2 August article on what your event logs reveal before you deploy agents ended on one observation: teams that get past the pilot stage use the mined model to decide which steps stay deterministic. What follows is the matching grid, five questions asked step by step, along with the data that lets you decide without waiting six months of production.

A six-step chain where the first two appear tripled in orange and the following four are sharp navy shapes, split by a vertical line.
The line between what stays code and what justifies probabilistic reasoning is drawn at the entrance of the process, not at its exit.

What reproducibility measurements actually show

A study published on arXiv in April 2026 by Abel Yagubyan asked the question in the plainest possible terms: does the same agent behave the same way twice? The protocol covers 19 tasks across five categories, six models from three providers, and 1,140 execution traces. The result comes down to two numbers. Tool sequence similarity, meaning the tendency to call the same functions in the same order, averages 0.87. Consistency of the arguments passed to those functions drops to 0.69. The gap between the two is wide and statistically very clear.

Three findings from that study matter more than the averages. First, ambiguity in how the task is phrased cuts argument consistency by 28% relative to a structured task, and that effect is larger than the effect of model choice, which does not reach significance. Picking a better model to stabilize an agent therefore addresses the second factor while ignoring the first. Second, 60% of behavioral divergence originates in the first two pipeline steps: what happens afterwards is largely determined by how the agent interpreted the initial request. Third, natural language output almost never matches across runs, under 5% exact match, even when tool sequences are identical. A regression test that compares text output is measuring noise.

The study also links stability directly to correctness: high structural consistency conditions reach 90.2% correct answers against 61.2% for low consistency ones. That is the usable part. You can measure whether a step needs an agent before you have any ground-truth data, simply by rerunning it twenty times and watching whether it does the same thing. A step that always does the same thing did not need an agent. A step that never does it the same way twice is not ready to run unsupervised.

A second piece of work, ReliabilityBench, published on arXiv in January 2026 by Aayush Gupta, measures degradation under production-like stress across 1,280 episodes in four business domains. Semantically equivalent rephrasings of the request alone move the success rate from 96.9% to 88.1%. API rate limiting turns out to be the most damaging fault of all those injected. One caveat about this paper: the models evaluated, Gemini 2.0 Flash and GPT-4o, belong to an earlier generation, and the absolute values have aged. The structural lesson holds: variance comes as much from the execution environment as from the model, and a single-pass benchmark measures neither.

The determinism test, five questions per step

The grid applies step by step, never to the whole process. A process is almost never entirely agentic or entirely deterministic; the decomposition is what produces reliability.

First question, input variability. Can you enumerate the shapes the input to this step takes? If the answer is yes, even with a long list, you are describing a routing table and you should write code. Probabilistic reasoning only earns its place once the list is open-ended and new cases show up unannounced. This question eliminates the most steps, and it is the one projects skip because it looks too simple.

Second question, reversibility. Can the action be undone, and within what window? A database write reversible in one transaction, an unsent draft, a ticket that can be reopened all tolerate a probabilistic decision. A payment, a customer-facing send, a deletion, a contractual commitment do not. The working rule is that the agent proposes and irreversible execution goes through code with explicit approval.

Third question, blast radius. How many systems and records does a single action touch? A decision affecting one case can be recovered from, a decision triggering a bulk action cannot. This connects directly to the governance frame described in the 23 July article on the agentic risk framework, with one difference: here it is not used to authorize or refuse the agent, but to decide whether this particular step should be code.

Fourth question, the need to explain. Will someone have to explain this decision to a third party, an auditor, a regulator, a customer, within a constrained deadline? If so, you need to be able to replay the decision identically. The data above shows that identical replay is not what an agent guarantees. A step subject to an explanation requirement belongs in code, or in code applying a rule the agent proposed upstream and a human froze.

Fifth question, cost of error against cost of verification. If checking the output costs as much as producing it, the agent saves nothing and merely relocates the work. Most pilots never run this calculation, and it often explains why a technically successful pilot never scales.

Five-card numbered diagram showing the five questions of the determinism test and the decision criterion attached to each.
The five questions of the test, applied step by step rather than to the whole process.

What the platforms say between the lines

Vendors selling agents are describing, with increasing precision, what their agents should not do. In its 7 July 2026 post, Google Cloud recommends transitioning the predictable parts of an agent workflow into deterministic code, and states an even stronger principle about guardrails: because those boundaries are implemented as deterministic constraints outside the agent's reasoning, they cannot be bypassed. The same text presents structured workflows, with code-level routing, conditional logic and state management, as the mechanism that produces predictability, with the agent operating inside that frame rather than replacing it.

Oracle announced on 14 July 2026 an extension of its AI Agent Studio for Fusion Applications, defending an architecturally adjacent thesis: agentic applications run natively inside the enterprise system, inheriting its business objects, workflows, approvals and auditability. Chris Leone, executive vice president of Applications Development, frames it as an explicit contrast: building disconnected AI automations and then trying to bolt on enterprise controls later is a fundamentally different approach. Read it for what it is, a commercial argument from an ERP vendor, but the argument is consistent with the reproducibility data: approvals, audit trails and transactional execution are precisely what does not tolerate variance.

This is not a new position. Anthropic already wrote, in its engineering note on building effective agents, that you should find the simplest solution possible and only increase complexity when needed, which might mean not building agentic systems at all. What separates 2024 from today is that we finally have measurements to arbitrate with, rather than a principle of restraint everyone endorses and nobody applies.

In the field, the pattern that recurs in deployments that hold is always the same. Routing, cost thresholds, permissions, transactional validations and logging move out of the agent's scope and stay as code. The agent keeps the interpretation of a freely worded request, the synthesis of heterogeneous information, the drafting, and the proposal of a plan someone else executes. Teams that fail did not pick the wrong model, they handed a probabilistic component decisions that had a known right answer all along.

What to start this week

Take the most advanced agent on your roadmap and decompose it into named steps, each with its input, its output and its side effect. If nobody on the team can produce that list within an hour, the model is not the problem.

Apply the five questions to each step and record exactly one decision beside each: code, agent, or agent under human approval. A step that gets an ambiguous answer stays code by default; the burden of proof is on the agent.

Run a consistency test before opening an architecture debate. Replay the same request twenty times on candidate steps and compare tool sequences and arguments, never the produced text. It costs a few hours and it settles the question faster than a meeting.

Add a perturbation to that test. Rephrase the same request five equivalent ways and rerun. That is the ReliabilityBench protocol reduced to its essentials, and it is what separates a demo from a production component.

Finally, for the steps that stay agentic, write down the deterministic constraint that bounds them: iteration cap, permission scope, cost threshold, list of forbidden irreversible actions. If that constraint lives in the system prompt, it does not exist.

Conclusion

The race to build agents has produced a design reflex that asks how to make the agent more reliable, when the useful question is which steps should never have been handed to it. The 2026 data finally gives you a way to decide without waiting for the incident: a step that replays identically did not need intelligence, a step that diverges on the second call is not ready, and the line between the two is read at the entrance of the process, where ambiguity either exists or does not. The best agent you ship this year will probably be the one with the fewest steps.

Sources: As of July 2026

Godwin AVODAGBE