CI test gate
what is ci test gate?
A CI test gate is a check that blocks a pull request from merging when tests fail. Its usefulness depends entirely on precision: a gate that also blocks on infrastructure problems and unreliable tests gets bypassed within weeks, and a bypassed gate is indistinguishable from no gate.
The distinction the gate has to make is between the product being broken and the check being unable to run. A network timeout, an expired credential and a browser that failed to start are all facts about your CI, not about the software under review.
Exit codes are how a runner says which is which. Zero for a clean run, one for a real failure, and a separate code for could-not-finish, so a workflow can block on the first and alert on the second without conflating them.
The second design question is who reads the result. A gate that only turns a check red makes people click into logs. Putting the verdict, the evidence and the likely cause in the pull request itself means the reviewer reads it where they already are.
in smolanalytics
The runner exits zero when everything passed, one when a test failed, meaning your app did not do what the sentence describes, and two when it could not finish. Stale recordings and runner errors never exit one. The result is a single comment on the pull request, edited in place on every push, naming the suspect file and quoting the evidence connecting it to the failure.
How it works shows where this fits in the loop, and the docs have the exact behaviour.