Can AI crawlers read JavaScript?

AI crawlers are the bots that AI companies send to fetch web pages, for model training and for live answers in tools like ChatGPT, Claude, and Perplexity. Most of them read only the HTML a server returns and do not execute JavaScript. Text, links, or structured data that a script adds after the page loads are invisible to them.

Google is the exception. Googlebot renders pages in a browser, so Gemini and AI Overviews can see script-loaded content. ChatGPT, Claude, and Perplexity cannot.

Which AI crawlers run JavaScript

The best public measurement comes from Vercel, which analyzed crawler traffic across its hosting network in late 2024. The finding was blunt: none of the major AI crawlers render JavaScript.

Crawler	Company	Renders JavaScript
GPTBot, OAI-SearchBot, ChatGPT-User	OpenAI	No
ClaudeBot	Anthropic	No
PerplexityBot	Perplexity	No
Meta-ExternalAgent	Meta	No
Bytespider	ByteDance	No
CCBot	Common Crawl	No
Googlebot (also feeds Gemini)	Google	Yes
Applebot	Apple	Yes

The crawlers do download JavaScript files, 11.5% of ChatGPT’s requests and 23.8% of Claude’s in Vercel’s data, but they do not execute them. A <div id="root"></div> stays empty.

How much traffic these crawlers send

AI crawlers are not a side issue. In the month Vercel measured, its network saw 569 million fetches from GPTBot, 370 million from Claude, 314 million from Applebot, and 24 million from PerplexityBot. Googlebot, for comparison, made 4.5 billion.

The same study found that ChatGPT’s and Claude’s crawlers hit 404 pages on about 35% of their fetches, against 8% for Googlebot. They guess URLs, follow stale links, and waste effort. Clean HTML, working redirects, and a current sitemap help them as much as they help Google.

Which sites lose content

The question is not the framework but where the content is put together: on the server, or in the visitor’s browser.

Setup	What AI crawlers see
Static HTML, WordPress, most shop systems	The full page
Next.js, Nuxt, Astro, SvelteKit with server rendering	The full page
A single-page app built with React or Vue, no server rendering	An empty shell
Content loaded from an API after page load	The page without that content
Tabs, reviews, and prices loaded on click or scroll	Whatever was in the HTML
JSON-LD added through a tag manager	No structured data
Text translated in the browser	The original language only

Mixed setups are the common case: a server-rendered page whose product details, reviews, or FAQ come from a script. The page looks complete in a browser, and a third of it is missing for ChatGPT.

How to check your own pages

Three ways, from quick to thorough:

View source, not inspect. View page source in the browser shows the raw HTML. If your main text is not there, crawlers do not get it. The Elements panel in DevTools shows the rendered page and misleads.
Turn JavaScript off. In Chrome DevTools, open the command menu and run “Disable JavaScript”, then reload. What remains is roughly what AI crawlers read.
Compare both versions. Fetch the HTML, render the page, and diff the text, links, and metadata. That is what our JavaScript SEO Check does.

How to fix it

The fix is to deliver the important content in the HTML response. In order of preference:

Server-side rendering or static generation. Every modern framework supports it: Next.js, Nuxt, SvelteKit, Astro, Angular with SSR. The page arrives complete, and JavaScript adds interactivity afterwards.
Prerendering. For a single-page app that cannot be rebuilt soon, prerender the public pages to static HTML at build time.
Hybrid. Keep the main content, navigation, metadata, and JSON-LD in the HTML. Load by script only the extras nobody needs to find: chat widgets, view counters, personalization.

Google calls “dynamic rendering”, serving bots a different version than users, a workaround, not a long-term solution. Server rendering serves everyone the same page and removes the problem.

Crawlers read pages; AI agents also click through them, which adds a second set of requirements: real buttons, named controls, a stable layout. See what an agent-friendly website is.

The 2 MB limit applies to the HTML too

Server rendering has one trap: pages that embed huge amounts of state in the HTML. Google’s crawler documentation states that for Google Search, Googlebot reads the first 2 MB of an HTML file, uncompressed, and indexes only that. A typical page is far below it. Framework pages that inline a whole product catalog as JSON can cross it, and then the text at the bottom of the page is cut off.

FAQ

Can ChatGPT read JavaScript websites? Not the parts that JavaScript loads. OpenAI’s crawlers read the HTML a server returns. If the content is in that HTML, ChatGPT can read it; if a script adds it, ChatGPT does not see it.

Does Google render JavaScript for AI Overviews? Yes. Gemini and AI Overviews use Google’s index, and Googlebot renders pages. Rendering can happen later than the first crawl, so content in the HTML still gets indexed faster.

Is my WordPress site affected? Usually not. WordPress renders pages on the server. Check page builder widgets, plugins that show customer reviews, and anything that loads content with a spinner.

Does llms.txt solve this? No. It is a file that points to content; it does not make script-loaded content readable, and Google has said it does not use it. Put the content in the HTML.

Do AI crawlers respect robots.txt? The major ones say they do, and our study of AI crawler blocking shows how many sites close the door on purpose. A page rendered only in the browser shuts them out by accident.

Check one of your pages with the free JavaScript SEO Check: it fetches the raw HTML, renders the same page in a real browser, and shows how much of the text, links, and metadata survive without JavaScript. For the broader view of how AI assistants read a site, see the best free AI visibility checkers. And if the content that matters sits behind a script, moving it to the server is work we do.