A recent audit of the Qwen2.5-1.5B model revealed that a token limit of 450 tokens was cutting off a significant portion of its responses, potentially skewing results. When the limit was increased to 1,024 tokens, most of the previously truncated responses were completed, with a notable increase in the rate of altered student quotes in the longer responses. The audit also introduced a new prediction that excerpts from English learners' essays would be more frequently altered compared to matched essays, and that longer replies would show a higher rate of quote changes. AI
IMPACT Highlights the critical importance of adequate token limits for accurate LLM output and research integrity.
RANK_REASON The item details a research audit of an LLM's behavior under different token limits and its impact on response fidelity. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
