End-to-end tests for Flask.
Flask apps are small on purpose, which is usually also why they have no tests: the whole thing is three routes and a template, and writing a browser harness for it feels absurd. A sentence is not absurd. Write one per route that matters.
$ npx smolanalytics test \
--url http://localhost:5000 \
--test "submitting the contact form shows a thank-you message"That first run needs no signup and no key. It opens a browser, does the thing, and prints the verdict.
how do I write end-to-end tests for a Flask app?
For a Flask app with no test suite, the cheapest useful coverage is one sentence per route that matters: npx smolanalytics test --url http://localhost:5000 --test "submitting the contact form shows a thank-you message". That first run needs no account, no key and no config file. An agent opens a real browser, reads the page through its accessibility tree, does what the sentence describes, and prints passed or failed. Nothing is added to requirements.txt and nothing is written to your repo, because nothing runs inside your app. The same command works in CI with --suite tests/ pointed at a staging URL, where it leaves one comment on the pull request saying what broke, edited in place, using the GITHUB_TOKEN GitHub Actions provides. A test that passes is recorded and replays afterwards with no model call at all, so a small suite stays cheap to run on every push. What it will not do is seed your database or construct a state for you: it uses whatever the environment is already in. The same walk through the app also writes and maintains your tracking calls in the analytics you already use, or the included engine can be your analytics if you run none. 14-day trial at Pro limits, no card, then $19/month.
The failure mode of a small Flask app is that nothing is checking it at all. There is no CI stage to add a test to, so the test never gets written, so a change to a template or a route breaks the one form the app exists for and nobody finds out until the emails stop. The economics of a sentence are different from the economics of a suite: three sentences is ten minutes, and they keep working when the app changes because there is no selector in them to update.
What this catches in Flask specifically is template and route drift. A url_for pointing at a renamed endpoint raises at render time on one page and not on the one you tested by hand. A form whose field name changed so request.form.get returns None and the handler stores an empty row while still redirecting to the success page — 302, then 200, and the whole thing looks fine from every angle except the one where you read the database. An agent that fills in the form and then confirms the thank-you message is a check on the visible outcome, and it goes red on the empty-row version only if the page actually says something different, which is worth knowing when you write the sentence: describe the thing the user should see, not the thing the server should do.
The second half is instrumentation, and small apps are where it is most often missing entirely. The agent has just used the form, so it knows the form exists, and it writes and maintains the tracking call in whichever SDK you already run — PostHog, Mixpanel, Amplitude, Google Analytics, Plausible or Segment. If you run none, the same engine can be your analytics: autocapture, funnels, retention and paths are included.
How it works walks the whole loop from the sentence to the comment on the pull request, instrumentation covers the second half — the tracking calls the same walk writes and maintains — and the free tools run without an account. The docs have the flags, the suite format and the CI step.
Pro $19/mo, 100 tested pull requests included, unlimited projects, after a 14-day trial at Pro limits with no card. full pricing, including overage →
One sentence, on your Flask app, tonight.
Point it at a URL that is already running and describe what should work. The first run uses the agent; once it passes it is recorded, so every run after that replays with no model at all — measured on our own site, 8.0s the first time and 1.4s the second.