End-to-end tests for Gatsby.
Gatsby resolves your data at build time, which means a whole category of breakage exists only in the built output and never in gatsby develop. Point a sentence at the deploy preview and an agent goes and uses the site that your visitors will get.
--- title: "A reader can find a post from the index" --- Open the blog index, click the first post, and confirm the post title and the author name are both on the page.
Point it at the deploy preview URL your host already built. There is no preview environment for us to create and nothing to provision.
how do I write end-to-end tests for a Gatsby app?
To test a Gatsby site end to end without writing test code, run npx smolanalytics test --suite tests/ --url against the deploy preview your host already builds, with one sentence per flow in a markdown file. This matters more on Gatsby than on a server-rendered stack because the interesting failures are created by the build: a GraphQL field that resolves to null and renders an empty author, a page createPages never generated so the index links to a 404, an image transform that produced no srcset. gatsby develop resolves data by a different path and shows you none of it, so the built site is the only honest thing to test. An agent opens a real browser, reads the page through its accessibility tree, clicks through the flow, and returns passed, failed, stale or errored. On a pull request one comment says what broke, edited in place, posted from your own CI runner with the GITHUB_TOKEN GitHub Actions provides. A passing run is recorded and replays afterwards with no model call at all. Nothing is written to your repo and no GitHub App is required. The same walk also writes and maintains your tracking calls in PostHog, Mixpanel, Amplitude, Google Analytics, Plausible or Segment. 14-day trial at Pro limits, no card, then $19/month.
The Gatsby failures worth catching are build-time failures that render as ordinary pages. A GraphQL query that quietly returns null for a field the template reads, so the post renders with an empty author. A page that was not created because createPages skipped it, so the link from the index is a 404 that only exists on the deployed site. A gatsby-plugin-image transform that failed and left the srcset empty. gatsby develop resolves data differently and shows you none of it. Something has to open the built site and look, and that is what a sentence like "click the first post and confirm the author name is on the page" makes happen.
The client-side router is the other one. Gatsby serves the first page as static HTML and then takes over navigation in JavaScript, so the page you reach by typing the URL and the page you reach by clicking the link are produced by two different code paths. The agent clicks, which is the path that is less tested and more used.
The instrumentation half is the same walk, and Gatsby sites are usually the ones with the least of it: a page-level analytics plugin counting pageviews and nothing at all on the newsletter signup or the contact form, which are the only conversions the site has. Because the agent has just used those forms, it knows they exist, and it writes and maintains the tracking calls inside the SDK you already run — PostHog, Mixpanel, Amplitude, Google Analytics, Plausible or Segment.
How it works walks the whole loop from the sentence to the comment on the pull request, instrumentation covers the second half — the tracking calls the same walk writes and maintains — and the free tools run without an account. The docs have the flags, the suite format and the CI step.
Pro $19/mo, 100 tested pull requests included, unlimited projects, after a 14-day trial at Pro limits with no card. full pricing, including overage →
One sentence, on your Gatsby app, tonight.
Point it at a URL that is already running and describe what should work. The first run uses the agent; once it passes it is recorded, so every run after that replays with no model at all — measured on our own site, 8.0s the first time and 1.4s the second.