If you want Google to send you traffic, it first has to read your pages. When it cannot, Google Search Console flags the problem, and learning how to fix crawl errors is one of the most useful skills a small business owner can pick up. The good news is that most of these errors are common, well understood, and fixable in an afternoon.
In this guide, we will walk through what crawl errors actually are, the ones you will see most often, and exactly how to clear them so Google can keep reading every page on your site. If you have not connected your site yet, start with our walkthrough on how to set up Google Search Console, then come back here.
What crawl errors actually are
Google finds your pages using an automated program often called a crawler or bot. It follows links, reads your XML sitemap, and requests each page the way a visitor's browser would. A crawl error simply means the bot tried to reach a page and something got in the way.
Crawl errors matter because a page Google cannot read is a page Google cannot rank. If a crawler hits a wall, that page (and sometimes the pages linked from it) can quietly drop out of search results. Fixing these problems is a core part of technical SEO, the behind-the-scenes work that makes everything else you do actually count.
Where to find crawl errors in Search Console
Google moved away from the old "Crawl Errors" report a while ago. Today the information lives in two main places:
- Pages report (under Indexing): this shows which URLs are indexed and, more usefully, which are not indexed and why.
- URL Inspection tool: paste any single URL to see exactly how Google sees it, when it was last crawled, and any errors it hit.
Open the Pages report and look at the "Why pages aren't indexed" table at the bottom. Each row is a reason, and clicking it lists the affected URLs. That table is your to-do list.
The most common crawl errors and how to fix them
404 (Not found) errors
A 404 means Google asked for a page that no longer exists. This is normal when you delete or rename pages, but a pile of 404s from real, linked pages is worth cleaning up.
- If the page moved, set up a 301 redirect from the old URL to the new one so visitors and Google land in the right place.
- If the page is gone for good and has no replacement, letting it return a 404 is perfectly fine. Google will drop it over time.
- Fix any internal links still pointing at the dead URL so you stop sending visitors into a wall.
Server errors (5xx)
A 500-series error means your server failed to respond when Google knocked. These are more urgent because they can affect real visitors too. Common causes are hosting overload, a plugin conflict, or a temporary outage. If you see a spike, check with your host and confirm the site loads for you. Our guide on what to do when your website is down covers the troubleshooting steps in order.
Redirect errors
Redirects are helpful until they loop or chain too many times. A redirect error usually means URL A points to B, which points back to A, or a long chain that never resolves. Map each redirect to its final destination in one hop, and remove any circular ones.
Blocked by robots.txt
Sometimes a page is fine, but a rule in your robots.txt file is telling Google to stay out. If an important page shows as blocked, open your robots.txt and remove the disallow rule that covers it. Be careful here, since one stray line can hide your whole site.
"Discovered - currently not indexed"
This one confuses a lot of owners. It means Google knows the page exists but has not gotten around to crawling it, often because the page seems thin or low priority. Strengthen the page with genuinely useful content, add internal links to it, and make sure it is in your sitemap. Our guide to getting indexed faster digs into this.
Tip: after you fix an error, do not just wait. Open the URL Inspection tool, paste the page, and click "Request Indexing." It nudges Google to recrawl sooner instead of waiting for its next scheduled visit.
A quick reference table
| What you see | What it means | What to do |
|---|---|---|
| Not found (404) | Page no longer exists | Redirect if moved, or leave it if truly gone |
| Server error (5xx) | Your server failed to respond | Contact your host, check plugins and uptime |
| Redirect error | Loop or too-long redirect chain | Point each URL to its final page in one hop |
| Blocked by robots.txt | A rule is blocking the crawler | Remove the disallow rule for that page |
| Discovered, not indexed | Known but not yet crawled | Add content, internal links, request indexing |
How to keep crawl errors from coming back
Fixing errors once is good. Preventing them is better. A few habits keep your site clean:
- Check Search Console monthly. A quick look at the Pages report catches problems while they are small. Add it to your website maintenance checklist.
- Keep your sitemap current. When you add or remove pages, make sure the sitemap reflects it so Google is not chasing ghosts.
- Redirect before you delete. Any time you change a URL, set the redirect first. This protects both rankings and visitors.
- Watch your internal links. A tidy internal linking strategy helps crawlers find every page and stops broken links from piling up.
If crawl health feels like one more plate to spin, that is exactly the kind of thing a done-for-you service handles in the background. But even solo, the checklist above will keep the vast majority of small business sites in good shape.
Frequently asked questions
How long does it take Google to clear a crawl error after I fix it?
It varies. After you fix a page and request indexing, Google often recrawls within a few days, though it can take a couple of weeks for the Pages report to update fully. The report reflects the last crawl, so a fixed error may linger in the display for a while even though the underlying problem is gone.
Do crawl errors hurt my rankings?
A few 404s on genuinely deleted pages will not hurt you. What hurts is when important pages cannot be crawled at all, whether from server errors, a robots.txt block, or redirect loops. Those pages simply cannot rank. Clearing errors on the pages you care about protects your visibility, which is why they matter for small business SEO.
Should I fix every single URL in the report?
No. Focus on real, useful pages you want in search. Old URLs you intentionally removed, spammy links from other sites, and test pages can stay as 404s. Trying to redirect every last one wastes time and can create new redirect chains. Prioritize the pages that earn you customers.