研究人员正在探索大型语言模型（LLM）如何管理上下文信息，新论文提出了不同的方法。一项研究介绍了“上下文语言模型”（CLMs），它们将上下文原生管理为文件，在浏览和多代理系统等任务中表现优于现有方法。另一篇论文研究了LLM中的“序数向量”，认为模型根据提及顺序而非实体名称来回忆事实。第三篇论文考察了LLM的“上下文绑定能力”，量化了模型在失去跟踪之前可以回忆多少次赋值，第四篇论文则探讨了如何使用LLM将任务相关的上下文编译到贝叶斯优化中。
AI
arXiv:2610.00910v1 Announce Type: cross Abstract: Human reasoning depends on how objects are related within propositions. \textit{How do relations organize the language representations of contextual contents?} We give an LLM a list of facts in its context (e.g., \emph{Alice eats …
arXiv cs.AI
TIER_1(CA)·Rulin Shao, Shannon Zejiang Shen, Junjie Oscar Yin, Yuetai Li, Minheng Wang, Hamish Ivison, Radha Poovendran, Nathan Lambert, Teng Xiao, Mike Lewis, Wen-tau Yih, Luke Zettlemoyer, Pang Wei Koh·
arXiv:2609.37725v1 Announce Type: new Abstract: We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows t…
arXiv cs.LG
TIER_1English(EN)·Zhongwei Yu, Sourabh Roy, Bin Cao, Xue Yan, Anjie Liu, Jun Wang·
arXiv:2609.36788v1 Announce Type: new Abstract: Incorporating rich task-relevant context, such as domain knowledge and external observations, is a key capability yet remains challenging for Bayesian optimisation (BO). Recently, practitioners have started to use large language mod…
arXiv cs.LG
TIER_1English(EN)·Manas Venkata Sai Ravulapalli, Samrath Singh Chadha·
arXiv:2609.30634v1 Announce Type: new Abstract: How many assignments can a language model recall before it loses track of which value belongs to which entity? We measure this limit using continuous recall curves for 12 models at or below 3B parameters and a threshold sweep over 3…
We introduce Context Language Models (CLMs), language models that natively manage their own context. We implement this by treating the context as a file and allowing the model to make unrestricted updates to this file. This allows the model to learn what is most important to main…
