Deterministic replay
what is deterministic replay?
Deterministic replay means re-running a test from a recorded sequence of steps rather than by reasoning about the page again. Because no model call is made, the run is fast, cheap and identical every time. The trade-off is that a recording is a snapshot: when the page moves, the replay stops fitting and something has to decide what happens next.
The economics are the point. Asking a model to work out how to use an application is slow and costs money on every run, which is fine once and untenable on every commit. A recording turns the expensive step into a one-off, so a stable suite runs at the speed of ordinary browser automation.
This is not a rare or proprietary idea, and any vendor implying otherwise is selling. Several agentic testing tools cache resolved steps and replay them without a model, and every tool that emits plain Playwright code gets the same property for free, because generated code has no model in it at runtime.
Where implementations genuinely differ is the failure mode. When a replay no longer fits, a tool can call it a failure, silently fall back to an agent and hide the cost, or report the mismatch honestly and then re-derive the flow. Only the last one gives you a number you can act on.
in smolanalytics
The first run of a test uses the agent and is recorded on success. Every run after that replays the recording with no model call at all; the published one-step run on the homepage replayed in 2.8 seconds. When a recording no longer fits, the run is reported as stale rather than failed, and the agent is woken to work the flow out again from the original sentence.
How it works shows where this fits in the loop, and the docs have the exact behaviour.