Deterministic Testing for Agentic Pipelines

Non-determinism in agentic pipelines is a structural feature to be designed around rather than a flaw to be debugged out of existence. It is a structural feature of how these systems work, and the testing discipline around them has to be built on that premise rather than around the hope of eliminating it.

Why non-determinism in agentic pipelines is structural

The unit of behavior in an agentic system is a trajectory, not a discrete output, covering the tools an agent chose to invoke, what it pulled from memory, and who it delegated a sub-task to along the way. That distinction matters more than it sounds like it should. A trajectory covers the tools an agent chose to invoke, what it pulled from memory, and who it delegated a sub-task to along the way. Conventional software testing was never built for that. It rests on a simple contract: input X produces output Y, every time, and a test either confirms that or it doesn't. Large language models break that contract at the model level, and agentic pipelines compound the break by layering tool selection, execution order, and multi-step reasoning on top of a foundation that was already probabilistic.

A joint paper from Arcadia Impact and Veraitech reviewed 240 documented testing and evaluation practices and found that established methods rest on eight assumptions about the systems they're evaluating, grouped into four clusters: specifiability, stability, composability, and supervisability. Agentic properties violate assumptions in every one of those clusters. That is a wide, structural failure, not a narrow edge case that a patch could fix.

Three sources of variability drive most of it. Tool selection is the first: the same goal can route through entirely different tools on different runs, and both routes might still land on a correct answer. Execution order is the second: steps can happen in a different sequence and the outcome can still look nominally right, which makes "did it work" a much fuzzier question than it is in deterministic software. The third is compounding, and it's the one with real teeth. The Alibaba "Grounded Scaling" paper works out the math: if a single step has per-step determinism δ that's less than 1, a chain of k steps succeeds with probability δ^k. That's exponential decay, not linear drift.

Layered on top of this, some parts of the system only take shape at runtime. Agents can pick sources and tools dynamically, and sub-agents can be bolted on after a system has already been certified, which opens a gap between the configuration that was tested and the one that's actually running in production. Systems that accumulate state compound the problem further: a system that has been running for a month with memory intact is not the same system that was characterized on day one, and evidence about its behavior ages unless someone updates it deliberately. None of this is a bug. It is a property of how these systems are built, and the only sound response is to engineer around it systematically rather than to keep chasing a determinism that was never actually available.

The limits of temperature=0 and a fixed seed

The most common fix teams reach for first is also the one that fails most quietly. Setting temperature to zero and fixing a seed feels like it should lock outputs down completely, and vendor documentation and empirical studies both confirm that it doesn't. OpenAI states that its seed parameter improves reproducibility without guaranteeing it, and Anthropic's documentation says the same thing about temperature 0.0 in its own words: results will not be fully deterministic. That's a direct account of how the serving stack actually behaves. It's an accurate description of how the serving stack actually behaves.

Understand the mechanism rather than just accepting it. Floating-point arithmetic is not associative, and modern inference engines parallelize computation across hardware in ways that change the order operations get summed in, which changes the result at the bit level. Batch size affects which kernel gets used, and hardware-specific optimizations introduce their own variance on top of that. Determinism, in other words, is a property of the entire serving stack, not something a request parameter can force into existence. Some frontier models released across 2025 and 2026 don't even expose a temperature parameter anymore, which quietly removes the illusion of control that many teams were leaning on.

Reasoning models add a second, separate layer of trouble. OpenAI's o-series, Claude's extended thinking mode, and similar reasoning-mode models sample their internal chain-of-thought before producing a final answer, so two runs at identical settings can take genuinely different reasoning paths and still land on different final answers. Best-of-k sampling, which many frontier benchmark setups use to report their best numbers, makes this worse by design: it samples k separate reasoning runs and keeps whichever one scores highest, which is non-determinism used deliberately rather than tolerated reluctantly.

The practical conclusion is blunt. Temperature=0 reduces variance at the margins. It does not remove it. Any evaluation methodology built on the assumption that a fixed seed produces a fixed output produces failures at the worst possible time in production, which is why the rest of the testing architecture has to assume variance will persist rather than trying to engineer it away entirely.

How widely the field currently ignores this problem

Given how well-documented this is at the vendor and research level, the gap in actual practice is striking. A review of 76 LLM-based code generation papers published over the last two years found that only 21.1% of them even considered non-determinism as a threat to their own experimental validity. That means the overwhelming majority of published benchmark results are drawing conclusions from single runs, or from methodologies that never account for run-to-run variance at all. Numbers built that way look precise. They aren't precise: numbers built from single runs or from methodologies that never account for run-to-run variance only look precise.

The pass@k versus pass^k distinction sharpens this further. When benchmarks report only pass@1 or pass@k, they're measuring whether an agent succeeded at least once across some number of attempts, and a meaningful share of that success reflects stochastic exploration rather than a capability the agent reliably possesses. That framing flatters systems in ways that don't survive contact with a production environment where an agent gets exactly one attempt.

Industry adoption tells a parallel story. McKinsey's State of AI 2025 report finds a substantial share of organizations already scaling agentic AI systems in production, with an even larger group actively experimenting, yet fewer than a third of them have clear architectural discipline around how their agents actually make decisions. Gartner, in August 2025, put a finer point on it: fewer than 5% of enterprise applications currently contain real, task-specific AI agents, even as the firm expects nearly every enterprise application to carry some form of embedded AI assistant by the end of 2025. A separate survey of 324 practitioners working in a specialized operations discipline for machine learning found a meaningful share reporting no agent deployment at all or only small pilots, with very few claiming anything like a fully integrated production rollout. The field, in short, is early. That earliness cuts both ways: testing discipline is either being built right now, while the stakes are still manageable, or it's being skipped in the rush to ship.

None of this is a failure of imagination. Teams know agents are powerful and are moving to deploy them. The failure sits at the architecture stage, where evaluation gets treated as a QA afterthought bolted onto a finished pipeline rather than a central engineering discipline built in from the start. A foundation built that way collapses once real traffic and real failure modes hit it.

The layered strategy: pinning components versus requiring probabilistic contracts

The way out of this starts with a distinction. Some parts of an agentic pipeline can be made genuinely deterministic. Others cannot, no matter how carefully they're configured, and pretending otherwise just produces brittle tests that pass in CI and fail in production. The core engineering move is to separate these two layers explicitly and apply a different testing philosophy to each.

The deterministic layer covers tool routing logic, request parsing, structured output schemas, and the order in which orchestration steps execute. These are conventional software components, even when they sit inside an LLM pipeline, and they can be pinned down and tested with the same assertion-based rigor that's always worked for deterministic code. The Sequential Pipeline pattern is a fixed-order chain of steps where each step's output feeds directly into the next, with no dynamic routing decision in between. That single design choice removes a major source of variance from the testable surface of the system. CrewAI's Flows mode is a working example of this idea built into a framework: an explicit, auditable mechanism for deterministic orchestration rather than leaving step sequencing to be decided on the fly.

But there's a caveat, because it's where a lot of teams fool themselves. Pinning the orchestration structure does not pin the model's behavior inside any given step. If a step in that fixed sequence contains an LLM call, the pipeline's shape stays deterministic while what happens inside each box remains variable. That's the boundary the whole strategy turns on.

Everything on the other side of that boundary, the LLM's reasoning itself, tool-selection decisions the model makes on its own, memory retrieval patterns, cannot be pinned to a single expected value. These components need something different: probabilistic contracts that define an acceptable distribution of outputs rather than one correct answer. Practically, that means deterministic-layer tests can stay simple and assertion-based, while non-deterministic-layer tests need semantic evaluators, statistical thresholds, and aggregation across multiple runs. One useful boundary mechanism ties the two together: hashing and versioning every prompt sent into the system, so that if the same prompt hash produces different answers, that divergence is visible immediately rather than buried in a support ticket weeks later.

Pinning the deterministic layer: infrastructure controls and their limits

For hosted models, a workable pinning stack starts with the basics: temperature set to zero along with the API's seed parameter where the provider supports one, and top_p set to 1 alongside that zero temperature. OpenAI responses also return a system_fingerprint value, and recording it lets a test suite re-run comparisons only when the fingerprint actually matches between runs, rather than assuming stability that isn't there. Pinning to a specific dated snapshot, something like "gpt-4o-2024-08-06" rather than a moving alias like "gpt-4o-latest", stops a provider's silent model update from quietly rewriting the pipeline's behavior overnight.

Self-hosted deployments carry the same logic further down the stack. Pinning the inference engine version, whether that's vLLM or TGI, along with the numerical precision (bf16 versus fp16), the batch size, and every sampling parameter, closes off several more sources of drift. Even after all of that, some drift should be expected, and the evaluator sitting on top of the pipeline needs to tolerate semantic equivalence rather than demand byte-for-byte identical output.

On the engineering side, separating deterministic agent logic (tool routing, parsing) from LLM output tests inside CI, using something as simple as Pytest markers, lets a team run the fast, deterministic tests on every single commit while reserving the slower, more expensive LLM evaluation tests for merges into main or a nightly run. Evaluation libraries themselves need the same version discipline applied to any other dependency: pinning tools like DeepEval or Ragas in a dedicated requirements-eval.txt file keeps a routine dependency bump from silently changing what "passing" means.

None of this eliminates variance. It reduces it, and it establishes the floor that the probabilistic layer needs to stand on. These controls are prerequisites for the harder work of building evaluation infrastructure that can handle genuine model-level uncertainty.

Building golden datasets and calibrated graders for the probabilistic layer

The shift here is conceptual before it's technical: stop checking for an exact output and start asserting on stable properties instead.

That starts with a golden dataset: a human-curated set of question, expected-answer, and context triples that together define the contract the agent has to satisfy. This dataset belongs in version control alongside the pipeline code, not off in a separate QA repository somewhere, because it is an engineering artifact in exactly the same sense the code is. It should also evolve. Failing production traces ought to flow back into the offline evaluation set so that every real failure hardens the test suite going forward, rather than getting fixed once and forgotten.

Code-based graders come first, wherever a deterministic check can settle the question. JSON schema validity is the simplest case: either the output parses correctly or it doesn't. Structural correctness checks against the pipeline's actual state graph confirm the system took a valid path rather than relying entirely on an LLM-judged rubric to catch a structural failure after the fact. Fact-presence checks round this layer out: does the response contain the required contract-level facts, independent of how it phrased them.

Model-based graders take over where judgment genuinely requires semantic understanding: faithfulness to source material, instruction following, hallucination detection, relevancy scoring. LLM-as-judge setups work here, but only with explicit, written criteria rather than an open-ended prompt asking a model to rate quality in the abstract. Calibration is the step that's easy to skip and expensive to skip. An LLM judge that hasn't been checked against a human-labeled gold set is itself a source of the same variance it's supposed to be measuring, which makes it circular rather than useful as a gate. Once a judge is calibrated, it needs minimum threshold scores set per metric, and those thresholds are the probabilistic contract in concrete, enforceable form.

Statistical discipline holds the whole thing together. Averaging evaluation scores across three or more runs absorbs non-deterministic variance in a way a single run never can, and reporting a mean alongside a standard deviation says something honest that a single score can't. This is where the pass@k versus pass^k distinction from τ-bench earns its place as the metric that actually matters for production: pass@k asks whether a system succeeded at least once across k attempts, while pass^k asks whether it succeeded every single time. Production contracts belong on the pass^k side of that line. A system that succeeds 80% of the time has a real failure rate, and production contracts should be written against pass^k, not pass@k.

Record-and-replay as the mechanism for turning failures into regression tests

Every technique above operates offline, against a dataset built ahead of time. Production failures behave like a different animal. A failure that appears in a live agentic pipeline depends on inference that can't be reproduced on demand, tools reading state that has already changed since the incident, and a multi-step trajectory that a simple re-run almost never repeats faithfully. Standard tracing tools record what happened. They don't make that recording testable, which means the most valuable failure data a team has, the one that actually happened in production, usually just sits in a log and gets forgotten.

Tracing and evaluation frameworks answer a narrower question: was this particular output acceptable. Neither lets an engineer take one recorded run, swap out a single component, and check whether that change fixes the failure while holding everything else exactly as it was. That gap is where deterministic replay comes in. It works by recording every source of non-determinism at its boundaries as the pipeline executes, then substituting those exact recordings back in during replay, turning what was once a non-reproducible incident into something that can be stepped through like a debugger. One requirement here is easy to overlook and expensive to skip: the replay engine has to intercept system clock calls and substitute the recorded timestamp, because a prompt containing a live date will produce different model behavior today than it did on the day of the original incident.

Chronicle, described in a paper posted in September 2026, is built around exactly this idea. It records an agent run at its non-deterministic boundaries as immutable envelopes and replays the run directly from that record. Its central mechanism is what the paper calls cut-point replay: a chosen subset of boundaries gets served straight from the recording while the complementary subset runs live against new code, which turns a single recorded incident into a regression test that can run inside CI on every future change. In benchmark testing across six recorded failures using simulated model boundaries, recording added 23 microseconds of overhead per boundary crossing, a negligible cost set against an assumed 300 millisecond model call, and full replay issued zero live model calls while remaining bit-stable across repeated runs. The work was presented at the AI Engineer World's Fair in 2026 and is scheduled to appear at the REALM workshop at EMNLP later that year.

The workflow this enables is straightforward to describe and hard to overstate the value of: a production failure gets captured, replayed to reproduce the exact trajectory that caused it, a code fix gets applied, cut-point replay confirms the fix without spending a single live model call, and the once-unreproducible failure becomes a permanent entry in the regression suite. That loop, from incident to permanent test, is what closes the distance between offline evaluation and the messy, non-deterministic reality of a system running in production. Golden datasets and calibrated graders tell a team what an agent should do before it ships. Record-and-replay tells them what it actually did when it broke, and makes sure it can't break the same way twice.