Can AI crawlers actually read your site?
ChatGPT, Claude and Perplexity fetch your HTML and read what the server sent. They run no JavaScript. So a site that ranks perfectly well on Google (whose crawler does render) can be a blank page to them. This checks in about ten seconds.
Free, no signup, and nothing to migrate. Whatever you use for analytics today — PostHog, Mixpanel, GA4, Plausible — keep it. None of them can see this, because these crawlers run no JavaScript, so there is nothing here to replace. It sits alongside.
questions
- Do AI crawlers run JavaScript?
- No. GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot fetch your HTML and read what the server sent. They do not boot your app. Googlebot does render JavaScript, which is why a single-page app can rank perfectly well in Google search and still be invisible to ChatGPT: the two look at genuinely different documents.
- My site is built with Lovable, Bolt, v0 or Vite. Is that a problem?
- It can be. Those tools default to client-rendered apps, where the HTML your server returns is an almost empty shell and the words arrive after JavaScript runs. That is exactly the case this scan is built to catch. The fix is per-framework (server rendering, static generation, or a prerender step) and it is usually a small change to how the route is built, not a rewrite.
- Does blocking GPTBot in robots.txt hurt me?
- It stops OpenAI reading your pages, which means you cannot be quoted from them. Plenty of people block AI crawlers on purpose and that is a legitimate choice, but it should be a decision, not a line inherited from a template. This scan tells you which of the seven AI crawlers your robots.txt currently turns away.
- Which crawlers does it check?
- Seven, by user agent: GPTBot (OpenAI's training and browsing crawler), OAI-SearchBot (ChatGPT Search's index), ChatGPT-User (ChatGPT fetching a page live for someone), ClaudeBot, Claude-User, PerplexityBot, and Google-Extended, which is Gemini grounding and is separate from Googlebot. The scan reports which of the seven your robots.txt currently turns away, and separately whether the HTML your server returns actually contains your words.
- What does this scan store?
- Nothing. It fetches the URL you type and that site's robots.txt, computes the checks, and returns them. There is no account, no cookie, and no tracking script involved in the scan itself.
- Is being readable enough to get recommended?
- No, and anyone claiming otherwise is selling something. Readability is the floor: a model cannot quote a page it never read. Whether it then names you depends on what it learned during training and what it retrieves at answer time. smolanalytics measures that separately, by asking the engines your buyers' questions on a schedule, judging each answer and recording it as an event on your own instance, with grounded and ungrounded engines kept apart rather than averaged and the exact model id on every row. Being exact about the boundary: that sampler spends a model call, so it draws on the plan's monthly AI allowance and does not run on a zero-dollar plan. This scan does not spend anything and needs no account.
Built by smolanalytics. There is one more thing you can run right now without an account: npx smolanalytics audit reads the repo you are standing in and names the user actions nothing is measuring, with the file and the line, and makes no network call at all. The paid product is the layer after that: about a day after you merge, the metric that actually moved is posted as a comment on the pull request that shipped it.