New hybrid NLI-LLM system detects evidence gaps in multi-hop QA

Researchers have developed StepGap, a novel hybrid system that combines Natural Language Inference (NLI) models with Large Language Models (LLMs) to identify evidence gaps in multi-hop question answering. This system categorizes these gaps into three types: Contradicted Claim, Irrelevant Evidence, and Missing Bridge, each suggesting a specific repair action. While StepGap's overall F1 score is comparable to LLM-only baselines, its structured approach offers greater interpretability and avoids error cancellation issues seen in purely LLM-based methods. When used to guide reinforcement learning, StepGap significantly improved the Exact Match score of the Qwen2.5-7B-Instruct model. AI

IMPACT This hybrid approach offers a more interpretable and robust method for improving multi-hop QA systems, potentially leading to more reliable AI assistants.

RANK_REASON The cluster contains a research paper detailing a new method for question answering. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Qwen2.5-7B-Instruct
• StepGap

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →