Google's Mueller says llms.txt cannot help LLMs differentiate between websites for discovery. He sees a narrow role once an agent is on a site. The file format
Google's Mueller stated that the llms.txt file format is not effective for helping large language models distinguish between websites during the discovery phase. He believes its utility is limited to a narrow role once an AI agent has already accessed a specific site. This file format has generated debate within the SEO community. AI
IMPACT The effectiveness of llms.txt for AI website differentiation remains uncertain, potentially impacting how AI agents discover and process online information.