Researchers have developed a new method called DarkLens to estimate how much large language models (LLMs) rely on context versus their parametric knowledge for generating responses. This technique projects hidden states onto a "dark space" of directions that interact weakly with the unembedding matrix, quantifying this reliance through a "Dark Matter Ratio" (DMR). When combined with a parametric knowledge score, DarkLens acts as a lightweight faithfulness detector, operating within a single forward pass. Evaluations on three benchmarks across six open-weight LLMs demonstrated that DarkLens matches or surpasses existing methods without requiring additional computational overhead. AI
IMPACT Provides a more efficient way to detect faithfulness hallucinations in LLMs, potentially improving their reliability in critical applications.
RANK_REASON The cluster contains a research paper detailing a new method for evaluating LLMs. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
