Are your HTML, CSS, and JavaScript files compressed before being sent to crawlers?
Compression shrinks text files by 70 to 90 percent over the wire; uncompressed sites burn the AI crawler timeout budget.
What this signal tests
We check whether your server returns text resources (HTML, CSS, JavaScript, JSON) with compression applied. The two accepted formats are Brotli, a modern algorithm signalled by Content-Encoding: br, and gzip, the older fallback. Compression typically shrinks text payloads by 70 to 90 percent, which means a faster download and a smaller chance of hitting a crawler timeout.
Why it matters for your visibility in AI
Uncompressed HTML is the easiest performance win you can give an AI crawler, and the most commonly overlooked. A 400 KB uncompressed page can shrink to 80 KB with Brotli, finishing four to five times faster on a typical connection. If your site takes eight seconds to send an uncompressed page, ChatGPT's crawler may give up and your content will not appear in answers, even though Google still indexes it. The same fix benefits mobile users on metered or slow connections, lowers your CDN egress bill, and improves Lighthouse score and Core Web Vitals. There is no downside for text content - compressed text decodes essentially instantly in any modern browser or crawler.
Pass criteria at a glance
| Criterion | Passes when |
|---|---|
| HTML has Content-Encoding br (preferred) or gzip; linked text resources compressed. Fail if any text resource >1KB uncompressed. |
How we test it
We send a GET request to your homepage with an Accept-Encoding header that advertises both Brotli and gzip support, the same way a real browser or compliant crawler does. We then read the Content-Encoding header on the response. We pass if the HTML is encoded with br (preferred) or gzip, and if linked text resources such as CSS and JavaScript files are also compressed. Any text resource above 1 KB sent uncompressed fails the check.
Show technical detection method
GET with Accept-Encoding: br, gzip; inspect Content-Encoding header on HTML and linked JS/CSS.
If your site fails: how to fix it
- If you use a CDN such as Cloudflare, Vercel, Fastly, or AWS CloudFront, enable Brotli in the CDN settings. Most CDNs offer it with a single toggle and apply it automatically to text MIME types.
- On nginx origins, install the ngx_brotli module and add brotli on; brotli_types text/html text/css application/javascript application/json; to your server block. Alternatively, enable gzip with gzip on; as a baseline.
- On Apache origins, enable mod_brotli (or mod_deflate for gzip) and add directives that target the same MIME types.
- Confirm compression is active by running curl -H 'Accept-Encoding: br, gzip' -I https://yourdomain.com/ and looking for Content-Encoding: br or gzip in the response headers.
- Do not compress already-compressed binary formats (JPEG, PNG, WebP, AVIF, video, PDF). Most servers skip these automatically, but check your config to avoid wasting CPU.
Quick facts
| Maturity | ESTABLISHED |
|---|---|
| Weight | medium |
| Category | Performance |
Primary sources
Related signals
Frequently asked questions
Will compression break anything?
No. Every modern browser, crawler, and AI bot supports gzip; Brotli is supported by every major browser since 2017 and by every compliant crawler. The Accept-Encoding header lets clients opt in, so servers only compress when the client requests it.
Should I prefer Brotli or gzip?
Brotli, when available. It typically produces files 15 to 25 percent smaller than gzip for the same content. Configure both and let the client choose - modern clients pick Brotli; older ones fall back to gzip.
Do I need to compress images?
No. JPEG, PNG, WebP, and AVIF are already compressed formats, and re-compressing them with Brotli or gzip wastes CPU without shrinking file size. Compression applies to text resources: HTML, CSS, JavaScript, JSON, XML, SVG.
Will fixing this also improve my Google ranking?
Indirectly, yes. Compression improves Largest Contentful Paint and Time to First Byte, both of which feed Google's Core Web Vitals ranking signals. The same fix helps AI crawlers, mobile users, and search ranking simultaneously.
Run your own scan
Run a free scan and see how your site grades across all 155 AI-readiness signals.