PulseAugur
EN
LIVE 10:45:30

Claude AI blocks user website despite permissive robots.txt

A user reported that Claude, an AI chatbot, has suddenly stopped being able to access their WordPress/WooCommerce website. This issue began after June 23, 2026, with Claude returning a "ROBOTS_DISALLOWED" error, even though the website's robots.txt file is permissive and other services like Googlebot and ChatGPT can access it without issue. The user suspects Anthropic may have implemented a domain-level block or is experiencing a caching issue, as the failed requests do not appear in their server logs. AI

IMPACT Potential impact on e-commerce sites relying on AI for product information and recommendations.

RANK_REASON User-reported issue with an AI chatbot's web access functionality.

Read on r/ClaudeAI →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

Claude AI blocks user website despite permissive robots.txt

COVERAGE [1]

  1. r/ClaudeAI TIER_2 English(EN) · /u/StereotypicalAussie ·

    Claude stopped being able to read my website, and it's driving me crazy

    <!-- SC_OFF --><div class="md"><p>I was happily using Claude to access my (very normal, built with wordpress and WooCommerce) website, to optimise, build out product pages, text, update descriptions, some design feedback etc)</p> <p>I run a small WordPress/WooCommerce business si…