Can AI crawlers actually read your site?

Questions

Do AI crawlers run JavaScript?

No. GPTBot, OAI-SearchBot, ClaudeBot and PerplexityBot fetch your HTML and read what the server sent. They do not boot your app. Googlebot does render JavaScript, which is why a single-page app can rank perfectly well in Google search and still be invisible to ChatGPT — the two look at genuinely different documents.

My site is built with Lovable, Bolt, v0 or Vite. Is that a problem?

It can be. Those tools default to client-rendered apps, where the HTML your server returns is an almost empty shell and the words arrive after JavaScript runs. That is exactly the case this scan is built to catch. The fix is per-framework — server rendering, static generation, or a prerender step — and it is usually a small change to how the route is built, not a rewrite.

Does blocking GPTBot in robots.txt hurt me?

It stops OpenAI reading your pages, which means you cannot be quoted from them. Plenty of people block AI crawlers on purpose and that is a legitimate choice — but it should be a decision, not a line inherited from a template. This scan tells you which of the seven AI crawlers your robots.txt currently turns away.

What does this scan store?

Nothing. It fetches the URL you type and that site's robots.txt, computes the checks, and returns them. There is no account, no cookie, and no tracking script involved in the scan itself.

Is being readable enough to get recommended?

No, and anyone claiming otherwise is selling something. Readability is the floor: a model cannot quote a page it never read. Whether it then names you depends on what it learned during training and what it retrieves at answer time — which is what the rest of smolanalytics measures, by asking the engines your buyers' questions on a schedule and recording what comes back.

Built by smolanalytics, which measures the other half: what the AI engines say about you, and what that traffic does after it lands.