LLM Numeracy Suffers with Underrepresented Scripts and Formats

A new research paper explores how large language models (LLMs) handle numerical reasoning when presented with diverse numeral scripts and formats. The study found that LLM accuracy significantly decreases when numerical inputs use underrepresented scripts or formats, even if the mathematical logic remains the same. Researchers demonstrated that specific prompting techniques, such as few-shot prompting and explicit numeral mapping, can substantially improve performance in these scenarios. These findings underscore a challenge in multilingual numerical understanding for LLMs and offer practical guidance for better numerical interpretation and generation across different scripts. AI

IMPACT Highlights challenges in multilingual numerical reasoning for LLMs and suggests prompting strategies for improvement.

RANK_REASON Research paper published on arXiv detailing findings about LLM performance on numerical tasks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Large language models
• LLM
• Varshini Reddy

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →