Stale recording
what is stale recording?
A stale recording is a recorded test run that no longer fits the application: a control it clicked cannot be found, or a step no longer applies. It is not a failure. A rename and a regression look identical to a replay, so reporting stale as a bug would mean claiming a product is broken every time somebody renames a button.
Record-then-replay testing works because a run that already passed can be repeated without asking a model to work anything out again. The cost of that speed is that a recording is a snapshot of a page that will move.
When it moves, there are two possibilities and the replay cannot tell them apart. Either the flow genuinely broke, or somebody renamed a button and the flow is fine. Any tool that colours this red is guessing, and it will guess wrong on every deliberate UI change, which is the most common kind of change there is.
The correct handling is to say what happened, in its own word, and then go and find out: re-run the flow with an agent that can look at the page again, and let that run produce the real verdict.
in smolanalytics
Stale is its own verdict, coloured amber and never red, and it is counted apart from failures in the pull request comment. When a recording stops fitting, the agent is woken to run the flow again from the sentence, and whatever that run finds is the verdict you get. The stale row exists so the reader knows the difference between a recording that needs refreshing and a product that needs fixing.
How it works shows where this fits in the loop, and the docs have the exact behaviour.