Many-shot CoT-ICL shows unstable scaling for reasoning tasks

Researchers have investigated the effectiveness of many-shot chain-of-thought in-context learning (CoT-ICL) for reasoning tasks, finding that standard many-shot approaches do not directly translate. Their study revealed that increasing CoT demonstrations can be unstable for non-reasoning models and primarily benefits reasoning-oriented LLMs. The research also indicated that similarity-based retrieval is effective for non-reasoning tasks but not for reasoning, and that performance variance increases with more CoT examples. To address these issues, they propose Curvilinear Demonstration Selection (CDS), an ordering method that improves performance by treating demonstrations as a structured curriculum for in-context test-time learning. AI

影响 Reframes in-context learning as test-time learning, suggesting new methods for ordering demonstrations to improve LLM reasoning.

排序理由 Academic paper detailing a new method for in-context learning. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• Curvilinear Demonstration Selection
• In-Context Learning
• Large Language Models
• Chain-of-Thought

• 论文
• 模型发布

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →