Does your robots.txt point AI crawlers to your sitemap?
A Sitemap line in robots.txt is the front door that helps AI crawlers find every page on your site.
What this signal tests
We check whether your robots.txt file includes a Sitemap directive that points to a valid, fully qualified sitemap URL. This single line is how most search and AI bots first learn the address of your sitemap, especially when they have never crawled your domain before and have no other discovery path.
Why it matters for your visibility in AI
Without a Sitemap directive, AI crawlers must discover your pages by following internal links from your homepage. Deep pages - case studies, blog archives, product detail pages, locations - are often missed entirely, which means they cannot be cited in AI answers about your business. For a freshly launched site, no Sitemap directive often means months of invisibility while crawlers slowly trip over individual pages. For a large site, it means a permanent long tail of content that AI systems never see, while your competitors who publish proper sitemaps get full coverage and steal share of voice in AI-generated answers.
Pass criteria at a glance
| Criterion | Passes when |
|---|---|
| >=1 absolute Sitemap: URL resolving to a valid sitemap. |
How we test it
We open your robots.txt file and look for one or more lines starting with the keyword Sitemap followed by a full URL. Each URL must include the protocol and the domain - not a relative path. We then visit each URL to confirm it loads and returns a recognisable sitemap document, either a single sitemap or a sitemap index that points to others.
Show technical detection method
Regex (?im)^\s*sitemap:\s*(https?://\S+) in robots.txt; each URL must be absolute and return a valid sitemap or sitemap index.
If your site fails: how to fix it
- Generate a sitemap for your site if you do not have one yet. WordPress plugins like Yoast or Rank Math produce one automatically; static-site frameworks usually have a built-in sitemap generator.
- Confirm the sitemap is publicly accessible at a stable URL such as https://yourdomain.com/sitemap.xml and that it loads without errors in a browser.
- Open your robots.txt file and add a line at the bottom that reads Sitemap followed by a colon, a space, and the full sitemap URL including https:// and the domain.
- If you have multiple sitemaps (for example one per language), add a separate Sitemap line for each, or point to a single sitemap index file that references them all.
- Save robots.txt and verify the new line is live by visiting /robots.txt in your browser, then re-run the AI Ready Test.
Quick facts
| Maturity | ESTABLISHED |
|---|---|
| Weight | high |
| Category | Crawlability |
Primary sources
Related signals
Frequently asked questions
Can I use a relative URL like /sitemap.xml in the Sitemap directive?
No. The standard requires an absolute URL with the full scheme and hostname. Relative paths are silently ignored by most crawlers, which means the directive will appear present but provide no value. Always use the full https:// URL.
What if my sitemap lives on a different subdomain or CDN?
That is fine - the Sitemap directive can point to any URL you control. Just make sure the URL resolves over HTTPS and returns a valid sitemap. Many sites host their sitemap on a CDN for performance, and this is perfectly acceptable.
Do I still need to submit my sitemap to Google Search Console?
Submitting in Search Console is still a good idea for traditional SEO reporting. But the robots.txt directive is the universal mechanism that every compliant bot - including AI crawlers that do not have submission portals - can read automatically. Do both for best coverage.
How many Sitemap lines can I have?
There is no fixed limit. You can list several sitemaps, one per line, all using the Sitemap keyword. Most sites either use one sitemap index file or list a handful of sitemaps directly. Keep the list maintainable and remove obsolete entries.
Run your own scan
Run a free scan and see how your site grades across all 155 AI-readiness signals.