CrawlCoach

See why pages are missing from Google and what to change. Connect your CrawlCoach account to read the pages Google left out, with Google's reason for each from your Search Console data, and your fix list: each fix with its page, why it matters, and the exact change. Then ask CrawlCoach to recheck the pages you changed. Without an account, check any public site's robots.txt, sitemaps, llms.txt, and page count, and whether robots.txt or a firewall blocks ChatGPT, Claude, or Perplexity from a page.

What you can do:

• Read your fix list: each fix with its page, why it matters, the exact change, and its status.
• List the pages Google left out, with Google's reason for each, and the pages it has.
• Recheck the pages you changed, so the fix status updates without waiting for the next scheduled check.
• Test pages against a live or draft robots.txt for Googlebot, Bingbot, and the AI crawlers.
• Check whether robots.txt or a firewall blocks ChatGPT, Claude, or Perplexity from a page, and whether its text is in the server HTML.
• Find sitemap problems, count a site's pages, and check or draft an llms.txt.

Account tools need a CrawlCoach account, which checks up to 50,000 URLs per site. The free checks work on any public site without one.