Craftora
Critical issueTechnical

Page Blocked by robots.txt: How to Fix It

A Disallow rule in robots.txt stops search engines from crawling this page.

Quick fix: Your robots.txt blocks this page. Remove or narrow the Disallow rule if you want it in search results.

Why it matters for SEO

Google cannot read blocked pages, so they rank poorly or show without a description. Important pages should never be blocked.

How to fix it

  1. 1Open yoursite.com/robots.txt and find the Disallow rule that matches this URL.
  2. 2Remove or narrow the rule, for example Disallow: /private/ instead of Disallow: /.
  3. 3Test the URL with the robots.txt report in Google Search Console.

Example

Blocks everything:  Disallow: /
Blocks only admin:  Disallow: /admin/

Free tools to fix it

Frequently asked questions

Why is my whole site blocked?

A leftover "Disallow: /" from development is the most common cause.

How fast does Google see robots.txt changes?

Google usually refreshes robots.txt within 24 hours.

Does your website have this issue?

Run a free SEO audit to check this and 40+ other issues in seconds.