SciEval: A Benchmark for Automatic Evaluation of K-12 Science Instructional Materials
A new paper proposes that LLM hallucinations stem not from a lack of knowledge, but from a failure in commitment, where models disperse probability mass across alternatives instead of concentrating on the correct answer. This phenomenon is observed to increase with model scale and is exacerbated by instruction tuning. Another paper introduces GAMMA, a framework for mixed-precision quantization that optimizes bit allocation for LLMs, significantly improving accuracy under memory constraints and outperforming existing methods on Llama and Qwen models. Additionally, a benchmark called SciEval has been developed to automatically evaluate K-12 science instructional materials, revealing that current mainstream LLMs perform poorly on this task without domain-specific fine-tuning. AI
IMPACT New research sheds light on LLM hallucination mechanisms and introduces novel methods for model optimization and evaluation, potentially improving reliability and efficiency.