Researchers have identified a phenomenon called "evidence bypass" in on-policy distillation (OPD) for scientific document question answering. This occurs when a student model retains an initially correct answer even after the supporting document is altered to support a different answer. Experiments on QASPER and PeerQA datasets using a Qwen3-4B student model showed that while standard OPD improves original-document accuracy, it also significantly increases evidence bypass. To address this, a new method called Evidence-Paired OPD (EP-OPD) was developed, which combines on-policy supervision with an answer-level margin to improve evidence responsiveness while largely preserving accuracy gains. AI
IMPACT This research highlights a critical limitation in current AI training methods, suggesting a need for more robust evaluation and training strategies to ensure AI models not only provide accurate answers but also reliably use supporting evidence.
RANK_REASON The cluster contains a research paper detailing a new phenomenon and method in AI model training. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
