LLMs outperform NLI models in scientific claim verification, fusion degrades performance

A new study explores the effectiveness of combining Natural Language Inference (NLI) models with Large Language Models (LLMs) for scientific claim verification. Researchers found that while LLMs like Qwen3 8B and Llama 3.1 8B showed promising results on benchmarks like SciFact and CliniFact, a proposed fusion method that required agreement between NLI and LLM models actually degraded performance. The study analyzed error patterns, suggesting that NLI models incorrectly overturning correct LLM predictions contributed to the decline. AI

IMPACT This research highlights the current limitations of LLMs in scientific claim verification and suggests that simpler fusion methods may not be optimal.

RANK_REASON The cluster contains an academic paper detailing experimental results on LLM performance for a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• BM25
• CliniFact
• DeBERTa-v3-base-mnli-fever-anli
• Lakshmi Narayanan Padmanabhan
• Llama-3.1:8b
• mistral:7b
• Qwen3_8B
• SciFact

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →