New Credal LLMs Improve Uncertainty Representation and Reduce Hallucinations

Researchers have introduced Credal Large Language Models (CLLMs) to address the issue of LLMs producing confident yet incorrect answers. Unlike standard LLMs that use a single predictive distribution, CLLMs employ an ensemble of LoRA adapters to create a credal set. This set exposes a range of plausible distributions, allowing for the derivation of commitment scores that better reflect uncertainty. The proposed methods, Credal Token Commitment (CTC) and Semantic Commitment Consistency (SCC), were evaluated on models like Gemma-2-9B, Llama-3.1-8B, and Qwen2.5-7B, showing improvements in question-answering accuracy and calibration while tracking hallucination rates effectively. AI

IMPACT Introduces a novel approach to uncertainty quantification in LLMs, potentially improving reliability and trustworthiness in AI applications.

RANK_REASON The cluster contains a research paper detailing a new methodology for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv stat.ML →

• ARC-Challenge
• Credal Large Language Models
• Gemma-2-9B
• Large language models
• Llama-3.1-8B
• LoRA adapters
• OpenBookQA
• Qwen2.5-7B
• TriviaQA

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →