How do I check if Google is crawling my site?

The fastest way to confirm Google activity is to check a specific URL in Search Console, compare its indexed snapshot with a Live Test, then review Crawl Stats for your host. If needed, verify in your server logs using Google’s official reverse DNS method. Don’t rely on a user-agent string alone, a Live Test as proof of indexing, or a missing example row as proof of zero requests.

Step 1: Inspect the exact URL in Search Console

Use the URL Inspection tool to see what Google last indexed for a page and when it last crawled it. The report is not live; it shows the most recently indexed version and includes key fields such as Last crawl, Crawled as (mobile or desktop), Crawl allowed?, Page fetch, and Indexing allowed? (URL Inspection tool).

• Open Search Console and paste the full canonical URL into the top “Inspect any URL” bar.
• Read the overall status. “URL is on Google” means eligible to appear, not a guarantee of showing in results. For definitive presence, search the exact URL in Google (URL Inspection tool).
• Expand Page indexing to find Last crawl, Crawled as, Crawl allowed? and View crawled page. These reflect the last indexed snapshot, not your current live page (URL Inspection tool).

What the indexed snapshot can tell you

• Last crawl: The date and time Google last crawled the URL; all inspection details are derived from that crawl (URL Inspection tool).
• Crawl allowed?: Whether robots.txt allowed crawling at the time Google last crawled the page (URL Inspection tool).
• Indexing allowed?: Whether noindex was detected at that time. Note: if robots.txt blocked the URL, Google cannot see noindex (URL Inspection tool).

Step 2: Run a Live Test to compare with what’s online now

Click Test live URL to fetch the page in real time. Use this to confirm that current robots.txt access, HTTP status and render are OK, and to validate fixes before requesting indexing. A valid Live Test does not guarantee indexing and may differ from the indexed version for several reasons (URL Inspection tool).

• Use Live Test to verify current accessibility, not to claim “Google crawled it.”
• After fixing issues confirmed in Live Test, you can Request indexing in URL Inspection (URL Inspection tool).

Step 3: Check Crawl Stats for site-level crawling patterns

Open Crawl Stats for your root property to see request volumes, response trends and Googlebot types across the host (Crawl Stats report).

• Availability and scope: Crawl Stats is only for root-level properties and aggregates requests for your domain (and subdomains for Domain properties). Data includes both HTTP and HTTPS for URL-prefix properties (Crawl Stats report).
• What to read: Total crawl requests, Average response time, Host status (robots.txt, DNS, connectivity), and breakdowns by Response code, File type, Crawl purpose, and Googlebot type (Crawl Stats report).
• Important nuances:
  • Example URLs are representative, not complete. If your URL isn’t shown, it doesn’t mean zero requests (Crawl Stats report).
  • Some requests might not be counted due to known coverage limitations; expect small differences versus server logs (Crawl Stats report).
  • When robots.txt is unavailable too long, totals may include crawls that were considered but not made, inflating counts versus your logs (Crawl Stats report).

Step 4 (optional): Verify in your server logs the crawls were genuine

If you want a second source of truth, validate that suspicious or high-volume requests were truly from Google. Do not trust a user-agent string alone; it can be spoofed. Google provides two verification options (Verify requests from Google crawlers):

• Manual reverse and forward DNS: Resolve the request’s IP to a googlebot.com or google.com hostname, then resolve that hostname back to the same IP.
• Automatic IP range matching: Match the request IP to Google’s published crawler IP ranges (CIDR lists) for common crawlers and special/user-triggered fetchers.

For a practical, privacy-safe way to test crawler access and log checks, see our guide: How to verify search and AI crawler access without exposing private pages. If you want background context on bots, read What is a Googlebot?

Common misreads to avoid

• “Live Test = indexed”: Live Test confirms real-time fetch and rendering; it does not prove the URL is indexed or will be indexed (URL Inspection tool).
• “Missing in examples = not crawled”: Example URLs in Crawl Stats are illustrative only, not exhaustive (Crawl Stats report).
• “User-agent says Googlebot, so it’s Google”: Always verify via DNS or published IP ranges (Verify requests from Google crawlers).

Quick checklist

• Inspect the exact URL in Search Console and note Last crawl, Crawled as, Crawl allowed?, and any fetch/indexing blockers (URL Inspection tool).
• Run a Live Test to confirm the page is currently fetchable and rendered correctly, then Request indexing if appropriate (URL Inspection tool).
• Open Crawl Stats to confirm host-level crawling patterns, response codes and Googlebot types; remember examples are not comprehensive and some requests may not be counted (Crawl Stats report).
• Optionally validate in logs that high-impact or suspicious requests are genuine Google using reverse/forward DNS or IP range matching (Verify requests from Google crawlers). For workflow details, see our crawler access guide.

FAQs

Can I make Google crawl faster?

You can’t force a higher crawl rate directly. Use sitemaps and solid site performance; monitor Host status and response times in Crawl Stats and address availability issues (Crawl Stats report). For crawl demand context, see What does the term "crawl budget" refer to in SEO?

Does “URL is on Google” guarantee it appears in search?

No. It means the URL is indexed and eligible, but appearance in results isn’t guaranteed. For a definitive check, search the exact URL on Google (URL Inspection tool).

Related guides

• What does the message "Crawled but not indexed" mean in Google Search Console?
• What is the cost of using Google Search Console?