RAG systems can use score thresholds to identify unanswerable queries

A recent analysis explored the effectiveness of using a simple score threshold on retrieval results to determine if a Retrieval-Augmented Generation (RAG) system can confidently answer a query based on its documentation. The study found that for the FastAPI documentation corpus using the MiniLM-L6-v2 embedding model, a cosine similarity score threshold of 0.40 successfully identified and refused approximately 24 out of 30 unanswerable questions while correctly answering 29 out of 30 answerable ones. While unrelated questions were easily filtered, near-domain questions presented a challenge due to overlapping scores, indicating that thresholds are corpus and model-specific and may require recalibration. The research also noted that traditional BM25 retrieval performed less effectively as a gatekeeper compared to cosine similarity. AI

IMPACT This research offers a practical method for improving the reliability of RAG systems by reducing hallucinations on unanswerable queries.

RANK_REASON Research paper detailing a method for improving RAG system accuracy. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• BM25
• FastAPI
• MiniLM-L6-v2
• OpenTelemetry
• retrieval-augmented generation

• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →