End-to-end tests for Shopify.
Shopify's checkout is not the part that breaks. The theme, the apps and the scripts wrapped around it are, and when they do, the store keeps taking traffic and stops taking money, quietly, until you look at the day's total.
--- title: "A shopper can buy one item with a test card" criticality: critical --- Open a product page, add the item to the cart, go to checkout, fill in the address, pay with the Bogus Gateway test card, and confirm the thank-you page shows an order number.
Run this against a development store with the Bogus Gateway enabled. We do not seed data, and nobody should be placing test orders on a live store.
how do I write end-to-end tests for a Shopify app?
For a Shopify store, the flow worth testing is add-to-cart through to the thank-you page, because that is where a silent break costs money the same day. Shopify's own checkout rarely fails; the theme, the apps and the scripts around it change constantly and are what break it — an app script that throws and stops the add-to-cart handler, a variant selector that stops updating the price after a section is duplicated, a cart drawer that opens empty because a theme update changed a section id, a discount field that accepts a code and does not apply it. All of them render perfectly. You write the flow as one sentence, run npx smolanalytics test --suite tests/ --url against a development store with the Bogus Gateway enabled, and an agent buys something in a real browser and confirms the order number appears. Use a development store rather than the live one: we do not seed data and nobody should be placing test orders on a real shop. The agent drives web browsers, so the Shop app is out of scope. A passing run is recorded and replays with no model call at all. The same walk also writes and maintains your add_to_cart and checkout tracking calls in the analytics you already use. 14-day trial at Pro limits, no card, then $19/month.
The economics here are different from every other page in this set. On a SaaS a broken flow costs you a signup you can win back next week; on a store a broken add-to-cart costs the day's revenue and you find out from the total. Everything that touches that path changes constantly and none of it is under version control in the way code is: a theme edit in the customizer, an app that injects a script, a Liquid section duplicated for a campaign, a discount that changes what renders on the cart page. Any of them can leave a beautifully rendered product page whose button does nothing.
The specific ones worth a sentence each: an app script that throws and stops the add-to-cart handler further down the page, a variant selector that stops updating the price after a section is duplicated, a cart drawer that opens empty because a theme update changed the section id it fetches, and a discount code field that accepts the code and does not apply it. Every one of them renders perfectly.
Two honest constraints. Use a development store with the Bogus Gateway rather than pointing this at a live checkout — we do not seed data, and a real store means real orders. And the agent drives web browsers, so the Shop app is out of scope.
The instrumentation half matters more on a store than anywhere else, because the events are the money: add_to_cart and checkout with an amount on them. The same walk that just bought something knows those steps exist, and it writes and maintains those tracking calls in the SDK you already run — PostHog, Mixpanel, Amplitude, Google Analytics, Plausible or Segment — so a theme change cannot quietly take your revenue events with it.
How it works walks the whole loop from the sentence to the comment on the pull request, instrumentation covers the second half — the tracking calls the same walk writes and maintains — and the free tools run without an account. The docs have the flags, the suite format and the CI step.
Pro $19/mo, 100 tested pull requests included, unlimited projects, after a 14-day trial at Pro limits with no card. full pricing, including overage →
One sentence, on your Shopify app, tonight.
Point it at a URL that is already running and describe what should work. The first run uses the agent; once it passes it is recorded, so every run after that replays with no model at all — measured on our own site, 8.0s the first time and 1.4s the second.