新方法增强大型语言模型在长上下文和多语言任务中的推理能力

报道来源 [4]

1. arXiv cs.CL TIER_1 English(EN) · Bingbing Wen · 2026-05-13 08:28

   STOP：低数据环境下长篇推理的结构化策略内剪枝

   Long chain-of-thought (Long CoT) reasoning improves performance on multi-step problems, but it also induces overthinking: models often generate low-yield reasoning that increases inference cost and latency. This inefficiency is especially problematic in low-data fine-tuning regim…

2. arXiv cs.AI TIER_1 English(EN) · Weitong Zhang · 2026-05-12 17:00

   OGLS-SD：基于策略的自蒸馏，结合结果引导的logit引导，用于LLM推理

   We study {on-policy self-distillation} (OPSD), where a language model improves its reasoning ability by distilling privileged teacher distributions along its own on-policy trajectories. Despite the performance gains of OPSD, we identify a common but often overlooked mismatch betw…

3. arXiv cs.CL TIER_1 English(EN) · André F. T. Martins · 2026-05-12 15:04

   结合On-Policy优化与蒸馏技术，实现大语言模型长上下文推理

   Adapting large language models (LLMs) to long-context tasks requires post-training methods that remain accurate and coherent over thousands of tokens. Existing approaches are limited in several ways: 1) off-policy methods such as supervised fine-tuning (SFT) and knowledge distill…

4. arXiv cs.CL TIER_1 English(EN) · Hinrich Schütze · 2026-05-10 14:06

   面向多语言推理的跨语言策略内蒸馏

   Large language models (LLMs) have achieved remarkable progress in mathematical reasoning, but this ability is not equally accessible across languages. Especially low-resource languages exhibit much lower reasoning performance. To address this, we propose Crosslingual On-Policy Se…