PulseAugur
EN
LIVE 21:35:45

AI crawlers struggle to access 27% of Show HN products

By PulseAugur Editorial · [1 sources] ·

A recent scan of 383 products launched on Show HN revealed significant accessibility issues for AI crawlers. Approximately 27% of these products were unreadable to automated systems, often due to client-side rendering without parsable server responses or aggressive bot filters. More surprisingly, 17% of the products actively blocked AI assistants like Claude, a measure frequently implemented by content delivery networks or web application firewalls rather than an explicit decision by the product founders. This invisibility means founders may misinterpret a lack of AI-driven demand as low product interest, as these failed requests do not appear in standard analytics dashboards. AI

IMPACT Highlights a critical gap in AI visibility for new products, potentially skewing market perception and demand signals.

RANK_REASON Analysis of product accessibility for AI crawlers, not a core AI release or research.

Read on dev.to — LLM tag →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

AI crawlers struggle to access 27% of Show HN products

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Analysis of product accessibility for AI crawlers, not a core AI release or research.
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
product, infra
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
47 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [1]

  1. dev.to — LLM tag TIER_1 English(EN) · Mike Viewfy ·

    We scanned 383 Show HN products. 27% were unreadable to crawlers, 17% blocked Claude (and nobody wrote that rule)

    <p>Most founders assume that if a page loads in a browser and shows up in Google, an AI assistant can read it too. We scanned 383 products from Show HN with a script we keep in the repo. 27% were unreadable to crawlers, and 17% were blocked from Claude outright.</p> <h2> 27% of 3…

RELATED ENTITIES

RELATED TOPICS