smolanalytics vs Shiplight.
An agent-native testing platform whose agents walk a real browser and leave plain YAML tests in your repository.
Shiplight or smolanalytics?
These two products agree about almost everything: agents write the tests, tests live as plain files, settled steps replay from cache without a model, and results land on the pull request. The differences are the price shape, $60 a month plus two usage meters against $19 flat on tested pull requests, and that our first run needs no account.
| smolanalytics | Shiplight | |
|---|---|---|
| How a test is written | One English sentence in a markdown file. No selectors, no page objects, no fixtures, no code to maintain. | Plain YAML in the repository, written by agents walking a real browser. |
| Price | $19/month with 100 tested pull requests included, then 10c each. 14-day trial at those limits, no card. The agent runs on your own Anthropic key, billed to you and never resold. | Free forever at $0 with a signup credit. Pro $60 a month of credit plus pay-as-you-go usage. Enterprise custom. |
| Cost of a run that passes | Recorded on the first pass, then replayed with no model call. | Cached steps replay without a model call; tokens go to self-healing rather than every run. |
| In your editor | An MCP server, connected once with npx smolanalytics connect. | Installs as an MCP server and as Skills; local automation needs no account. |
| Where the verdict lands | A comment on the pull request, edited in place, plus a project page holding the history. | Results land on the pull request. |
where Shiplight is the better pick
Shiplight publishes far more about the category than we do and has a free tier that is free forever rather than a trial. Their usage pricing is honest about where the money goes, tokens and compute, which is the right shape for anyone whose volume is spiky.
where smolanalytics is
A flat number you can predict. $19 a month with 100 tested pull requests included means the bill does not move when a refactor makes every recording stale for a week. We also keep stale and failed apart as separate verdicts with separate colours, which is the difference between a suite people read and a suite people mute.
How it works walks the loop from the sentence to the comment on the pull request, and the docs have the flags, the suite format and the CI step.
Pro $19/mo, 100 tested pull requests included, after a 14-day trial at Pro limits with no card. full pricing, including overage →
One sentence, on your app, tonight.
Point it at a URL that is already running and describe what should work. The first run uses the agent; once it passes it is recorded, so every run after that replays with no model at all.