A new research paper explores the distinction between task competence and instruction following in small language models. The study found that while models generally improve in both areas with scale, smaller models often ignore conflicting instructions, maintaining task accuracy but failing to adhere to user requests. This suggests that increased task capability does not automatically translate to reliable control over model behavior, and standard accuracy metrics can mask instruction-following failures. AI
IMPACT Highlights the need for better evaluation metrics beyond standard accuracy to ensure reliable control over LLM behavior.
RANK_REASON Research paper analyzing LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
