the analytics ai that cannot make up your numbers

can AI analytics be trusted? most of the time, no, because the AI writes the query and the number comes out of a language model that is happy to guess. smolanalytics is built the other way around: the AI never writes sql and never computes anything. it picks one of a fixed set of deterministic reports, that report returns the real number or nothing, and a CI test asserts the number you get over MCP equals the /v1 api equals the dashboard, byte for byte, or the build fails. so the number cannot be made up.

do AI analytics tools hallucinate?

yes, and it is worse than people think, because the hallucination looks correct. the standard pattern is text-to-sql: you ask a question, the model writes a sql query against your event tables, it runs, a number comes back. that number looks authoritative. it has a chart next to it. but the model guessed which table, guessed the join, guessed whether to count distinct users or raw events, guessed the timezone on your date filter. any one of those is a silent wrong answer, and you have no way to know from looking at it.

i did not want to ship that. an analytics tool that is confidently wrong is worse than no tool, because you will make a decision on the fake number and never find out. so smolanalytics is designed so the model is not allowed near the arithmetic.

how does smolanalytics stop the AI from making up numbers?

the AI never touches your data. it does one job: it reads your plain-english question and picks which pre-built report to run. that is the entire scope of the language model in the system. the reports themselves are ordinary deterministic go code that queries your events the same way every time, funnels, retention, top pages, drop-off, channel breakdown, defined events. the model chooses the report and the scope (which dates, which segment). the report computes. if there is no report that answers your question, you get "i cannot answer that from a report" instead of a plausible fabrication.

this is the whole trick and it is boring on purpose. text-to-sql fails because you handed a generative model a compute job. smolanalytics hands it a routing job. routing to the wrong report is visible and rare. inventing a number is impossible, because the model does not have a code path that produces numbers.

what is the "computed_by" receipt?

every answer from the ask bar ships a receipt that names the exact report and scope that produced it. you ask "where do people drop off?" and along with the number you get a computed_by block: which deterministic report ran, over which date range, which segment, which events. it is not a vibe. it is the provenance of the number, attached to the number.

this matters because it makes the answer checkable by a human in two seconds. you can look at the receipt, see it ran the funnel report over last 30 days on all traffic, and go run that same report yourself in the dashboard. same number. if the receipt said something you did not expect, that is your signal, not the number being quietly wrong.

what is the CI agreement test?

there is a test in the repo that fails the build if the three ways of getting a number ever disagree. this is the part i am proudest of. an analytics product has three surfaces that should all report the same thing: the AI answer over MCP, the raw /v1 json api, and the dashboard ui. in most tools these are three separate code paths that drift. one gets a bug fix, the others do not, and now your api says 1,204 signups and your dashboard says 1,197 and nobody notices for a month.

smolanalytics has a test that asserts they are identical, byte for byte, on the same fixture data. the AI's computed number, the api response, the dashboard's number: equal, or the build is red and the release does not ship. you can read it here:

github.com/Arjun0606/smolanalytics/blob/main/internal/api/agreement_test.go

that is not marketing copy. it is a file you can open. the claim "our numbers cannot drift" is either true or the test is passing under false pretenses, and you can go check the test. that is the bar i wanted for "trustworthy": not "trust us", but "here is the assertion, and it runs on every commit."

what about the tracking plan and drift over time?

the same distrust extends to your instrumentation, so that is version-controlled too. you keep a tracking plan as a committed file. plan check --code reads your repo statically and tells you where the code and the plan disagree. a traffic-based check tells you which planned events are not actually firing in production, and which live events are not in the plan. plan sync reconciles them. so the drift gate is not just "is the number right today", it is "is the thing being measured still the thing you agreed to measure." both are enforced, both are in git.

why does this let you use your own model for free?

because the model does so little, it can be small and yours. the AI is a router, not a calculator, so you point smolanalytics at your own model over MCP (Cursor, Claude Code, whatever you run) and the intelligence is free to you. you are not paying a per-query analytics-AI tax, and your event data is not being shipped to someone else's model to be processed. the deterministic reports do the real work locally, on your data, in one go binary.

smolanalytics is open source, MIT, a single binary you can self-host free and own your data. hosted is $9 for solo up to $499 for business, flat, with importers from PostHog, Mixpanel, Umami and CSV. or just try the ask bar on live data, no signup, at smolanalytics.com, and check the receipt on whatever it tells you.