English(EN)A Tool in the Forward Pass: Frozen Tape Modules as Layers of a Small Language Model
新研究解决大语言模型安全、效率和能力恢复问题
作者PulseAugur 编辑部·[23 个来源]·
近期研究探索了增强大语言模型（LLM）和多模态大语言模型（MLLM）的新颖方法。其中一种方法 MLLMEraser，通过在不更新参数的情况下引导激活，实现了多模态大语言模型的测试时遗忘，在遗忘指定内容的同时最小化效用损失方面优于现有方法。另一项研究引入了一个用于规范引导生成的框架，将大语言模型与符号推理相结合，以确保输出符合形式规范。此外，研究正在通过可训练向量量化和渐进式剪枝等技术推进大语言模型的有效压缩，旨在降低内存使用量并加快推理速度，同时保持模型质量。同时，也在努力通过增加有害和无害嵌入之间的分离度来提高大语言模型的安全性，并恢复在微调过程中丢失的通用能力。
AI
arXiv:2510.04217v4 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and har…
arXiv:2610.41968v1 Announce Type: new Abstract: Weight pruning can reduce the storage and inference cost of large language models (LLMs), but preserving quality at high sparsity levels remains challenging. Existing methods face a quality-cost trade-off in this regime. One-shot pr…
arXiv:2610.41511v1 Announce Type: new Abstract: Large language models (LLMs) can generate structured outputs, but existing constrained-decoding methods mainly enforce syntax and cannot directly enforce satisfaction of a formal specification. We introduce a neuro-symbolic, specifi…
arXiv cs.CL
TIER_1English(EN)·Nawar S. Alseelawi, Mustafa S. Aljumaily·
arXiv:2610.40893v1 Announce Type: new Abstract: Fairness benchmarks for large language models (LLMs) are overwhelmingly English-born and American in their social categories; translated versions carry those categories into societies where they do not apply, and almost none fix the…
arXiv:2610.41147v1 Announce Type: new Abstract: Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to p…
arXiv:2603.20206v2 Announce Type: replace Abstract: Safety alignment does not fully prevent large language models (LLMs) from producing harmful content under adversarial attacks. We observe that existing attack methods typically move harmful embeddings toward regions of harmless …
arXiv:2610.41445v1 Announce Type: new Abstract: Membership inference aims to determine whether a particular example was included in a model's training data. Such membership signals are often broadly described as memorization, while the underlying mechanisms can involve both how i…
arXiv:2610.41822v1 Announce Type: new Abstract: Vector Quantization (VQ) is a rapidly evolving technique for compressing Large Language Model (LLM) weights, reducing memory usage, and speeding up inference. However, extreme compression deteriorates model performance, which is cha…
Calling a tool through generated text adds decoding steps, even when the computation itself is short. We study an alternative: a small neural module that runs inside a language model's forward pass. A learned writer turns hidden states into a symbolic tape, a short sequence of di…
arXiv cs.AI
TIER_1English(EN)·Lingyun Guo, Ali Bereyhi·
arXiv:2610.37932v1 Announce Type: new Abstract: Large language models (LLMs) have emerged as potential tools for numerical optimization. Despite promising results, their intrinsic optimization capabilities and limitations remain poorly understood, particularly in high dimensions.…
arXiv:2610.37681v1 Announce Type: cross Abstract: Large reasoning models have recently demonstrated remarkable capabilities in solving complex tasks, where supervised fine-tuning (SFT) on long chain-of-thought data serves as a crucial foundation for eliciting and enhancing their …
arXiv:2610.37684v1 Announce Type: cross Abstract: Large Language Models (LLMs) have rapidly become important tools for science, education, medicine, and professional work. Despite their apparent simplicity from the user's perspective, effective and responsible use requires an und…
arXiv cs.AI
TIER_1English(EN)·Shaoliang Yang, Jun Wang·
arXiv:2610.38096v1 Announce Type: cross Abstract: When a language model returns engineering quantities with the status "solved", software that trusts it passes them on, even when the requested design is physically impossible. Such failures could reflect poor calculation, how the …
arXiv cs.AI
TIER_1English(EN)·Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune·
arXiv:2610.38917v1 Announce Type: cross Abstract: Large language models (LLMs) have rapidly become a default source of help for students in science and engineering courses. In foundational engineering mechanics courses such as statics, students routinely use these tools to check …
arXiv:2610.39923v1 Announce Type: cross Abstract: As language models (LMs) have improved, an increasingly common means of evaluating their linguistic competence is through explicit prompting (e.g., "Which sentence is more acceptable?"). Prior work indicates that such prompting un…
arXiv cs.AI
TIER_1English(EN)·Mingzi Cao, Xi Wang, Nikolaos Aletras·
arXiv:2610.40091v1 Announce Type: cross Abstract: Effective multi-domain adaptation of large language models requires prioritizing relevant domain knowledge while minimizing interference from irrelevant domains. Existing approaches commonly use Low-Rank Adaptation (LoRA) to train…
arXiv cs.AI
TIER_1English(EN)·Jaden Park, Jungtaek Kim, Jongwon Jeong, Robert D. Nowak, Kangwook Lee, Yong Jae Lee·
arXiv:2604.13151v2 Announce Type: replace Abstract: Language Model (LM) agents are increasingly used in complex decision-making tasks, from AI coding to physical AI, where they must both explore a search space and exploit acquired knowledge effectively. Although exploration and e…
arXiv cs.AI
TIER_1English(EN)·Jinane Bazzi, Mariam Rakka, Fadi Kurdahi, Mohammed E. Fouda, Ahmed Eltawil·
arXiv:2604.11512v2 Announce Type: replace-cross Abstract: The growing demand for deploying Small Language Models (SLMs) on edge devices, including laptops, smartphones, automotive and robotic computing platforms, and high-performance embedded systems, has exposed fundamental inef…
Reading "While the man hunted the deer ran into the woods" requires giving up the analysis in which "the deer" is the object of "hunted". People who have just read such a sentence still produce more transitive structures, so the abandoned analysis primes later language. We ask wh…
arXiv cs.CV
TIER_1English(EN)·Samuel Dawit Assefa, Jae Won Cho·
arXiv:2610.38296v2 Announce Type: replace Abstract: Language-model answers are often treated as a visible record of the internal decision that produced them. We test this assumption in conflicting-context question answering, where two supplied documents support incompatible answe…
Medium — MLOps tag
TIER_1English(EN)·Vijay Dasari·
<blockquote> <p><strong>TL;DR</strong>: I built a language model <strong>from scratch, end to end, on free GPUs</strong>: a 125.9M-parameter model that writes in a register you choose, with slang, emoji, abbreviation, elongation and formality as five dials. The idea is simple and…
<!-- SC_OFF --><div class="md"><p>Cloud models have a very big problem which is very hard to solve, and it is not technological.</p> <p>The biggest issue with cloud providers is that nearly all tech companies face, but worse: they want to rip the benefits of language models, but …
