pgvector HNSW filtering bug causes RAG systems to return zero results

A developer encountered a critical issue with retrieval-augmented generation (RAG) systems using the pgvector extension in PostgreSQL. The problem stems from how pgvector's Hierarchical Navigable Small World (HNSW) index filters results: the approximate index selects candidate vectors first, and then PostgreSQL applies the WHERE clause. This means that for smaller datasets or tenants (like one with 4,000 chunks representing 2% of a 200,000-chunk table), the WHERE clause can filter out all candidates, resulting in zero returned rows even when a LIMIT clause is present. The developer suggests solutions such as using iterative index scans, partial indexes, or partitions, and recommends logging retrieval queries that return fewer rows than requested to detect such issues. AI

IMPACT This bug can cause RAG systems to fail silently, leading to incorrect information being presented to users and impacting the reliability of AI-powered applications.

RANK_REASON The item details a specific technical bug and its resolution within a database extension used for AI applications.

Read on dev.to — LLM tag →

• Google Maps
• Hierarchical Navigable Small World graphs
• LIMIT+
• pgvector
• PostgreSQL
• Psql
• retrieval-augmented generation

• infra
• product

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →