New framework MissDiag evaluates KGQA and KG-RAG robustness to missing knowledge

Researchers have introduced MissDiag, a new framework designed to evaluate the robustness of knowledge graph question answering (KGQA) and knowledge-graph-based retrieval-augmented generation (KG-RAG) systems when faced with incomplete knowledge. Unlike previous methods that provide aggregate scores, MissDiag applies structured missingness interventions to support graphs while keeping questions and answers fixed. This approach allows for a more detailed understanding of how different types of missing evidence impact system performance and evaluation protocols, revealing that loss of adjacent evidence causes the most significant degradation. AI

IMPACT Provides a more interpretable method for diagnosing and stress-testing KGQA and KG-RAG systems under incomplete knowledge conditions.

RANK_REASON The item describes a new diagnostic evaluation framework for KGQA and KG-RAG systems published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• KGQA
• KG-RAG
• MissDiag
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →