Researchers have developed a new training objective called Smooth Maximum Mean Discrepancy (SMMD) to improve the numerical precision of large language models (LLMs). Standard cross-entropy training treats numerical tokens as categories, ignoring their inherent value structure. SMMD addresses this by incorporating value-distance kernels and graph-based smoothness, aligning predicted distributions with target values and encouraging local consistency. Evaluations across various LLM and vision-language model backbones on tasks like mathematical reasoning and chart question answering show SMMD consistently outperforms existing methods. AI
IMPACT This new training objective could lead to more reliable LLMs for tasks requiring numerical precision, impacting fields like scientific research and financial analysis.
RANK_REASON The cluster contains a research paper detailing a new method for improving LLM performance.
- Chart question answering
- clock-time recognition
- cross entropy
- large language models
- LLMs
- mathematical reasoning
- Smooth Maximum Mean Discrepancy
- value-distance kernels
- vision-language model
- kernel graph
- Kernel-matching pursuits with arbitrary loss functions
- Maximum Mean Discrepancy
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
