New research tackles LLM safety, efficiency, and capability recovery
ByPulseAugur Editorial·[23 sources]·
Recent research explores novel methods for enhancing large language models (LLMs) and multimodal LLMs (MLLMs). One approach, MLLMEraser, enables test-time unlearning in MLLMs by steering activations without parameter updates, outperforming existing methods in forgetting designated content with minimal utility degradation. Another study introduces a framework for specification-guided generation, combining LLMs with symbolic reasoning to ensure outputs adhere to formal specifications. Additionally, research is advancing techniques for efficient LLM compression through trainable vector quantization and gradual pruning, aiming to reduce memory usage and speed up inference while maintaining model quality. Efforts are also underway to improve LLM safety by increasing the separation between harmful and harmless embeddings and to recover general capabilities lost during fine-tuning.
AI
IMPACT
These advancements offer potential improvements in LLM safety, efficiency, and adaptability, addressing key challenges in their deployment and performance.
RANK_REASON
Multiple research papers published on arXiv detailing new techniques for LLMs and MLLMs.
arXiv:2510.04217v4 Announce Type: replace Abstract: Multimodal large language models (MLLMs) have demonstrated remarkable capabilities across vision-language tasks, yet their large-scale deployment raises pressing concerns about memorized private data, outdated knowledge, and har…
arXiv:2610.41968v1 Announce Type: new Abstract: Weight pruning can reduce the storage and inference cost of large language models (LLMs), but preserving quality at high sparsity levels remains challenging. Existing methods face a quality-cost trade-off in this regime. One-shot pr…
arXiv:2610.41511v1 Announce Type: new Abstract: Large language models (LLMs) can generate structured outputs, but existing constrained-decoding methods mainly enforce syntax and cannot directly enforce satisfaction of a formal specification. We introduce a neuro-symbolic, specifi…
arXiv cs.CL
TIER_1English(EN)·Nawar S. Alseelawi, Mustafa S. Aljumaily·
arXiv:2610.40893v1 Announce Type: new Abstract: Fairness benchmarks for large language models (LLMs) are overwhelmingly English-born and American in their social categories; translated versions carry those categories into societies where they do not apply, and almost none fix the…
arXiv:2610.41147v1 Announce Type: new Abstract: Fine-tuning improves a language model's performance in a target domain but can weaken its general capabilities. Existing methods often face a trade-off between general capabilities and domain specialization, making it difficult to p…
arXiv:2603.20206v2 Announce Type: replace Abstract: Safety alignment does not fully prevent large language models (LLMs) from producing harmful content under adversarial attacks. We observe that existing attack methods typically move harmful embeddings toward regions of harmless …
arXiv:2610.41445v1 Announce Type: new Abstract: Membership inference aims to determine whether a particular example was included in a model's training data. Such membership signals are often broadly described as memorization, while the underlying mechanisms can involve both how i…
arXiv:2610.41822v1 Announce Type: new Abstract: Vector Quantization (VQ) is a rapidly evolving technique for compressing Large Language Model (LLM) weights, reducing memory usage, and speeding up inference. However, extreme compression deteriorates model performance, which is cha…
Calling a tool through generated text adds decoding steps, even when the computation itself is short. We study an alternative: a small neural module that runs inside a language model's forward pass. A learned writer turns hidden states into a symbolic tape, a short sequence of di…
arXiv cs.AI
TIER_1English(EN)·Lingyun Guo, Ali Bereyhi·
arXiv:2610.37932v1 Announce Type: new Abstract: Large language models (LLMs) have emerged as potential tools for numerical optimization. Despite promising results, their intrinsic optimization capabilities and limitations remain poorly understood, particularly in high dimensions.…
arXiv:2610.37681v1 Announce Type: cross Abstract: Large reasoning models have recently demonstrated remarkable capabilities in solving complex tasks, where supervised fine-tuning (SFT) on long chain-of-thought data serves as a crucial foundation for eliciting and enhancing their …
arXiv:2610.37684v1 Announce Type: cross Abstract: Large Language Models (LLMs) have rapidly become important tools for science, education, medicine, and professional work. Despite their apparent simplicity from the user's perspective, effective and responsible use requires an und…
arXiv cs.AI
TIER_1English(EN)·Shaoliang Yang, Jun Wang·
arXiv:2610.38096v1 Announce Type: cross Abstract: When a language model returns engineering quantities with the status "solved", software that trusts it passes them on, even when the requested design is physically impossible. Such failures could reflect poor calculation, how the …
arXiv cs.AI
TIER_1English(EN)·Evangelos Apostolakis, Erfan Hamdi, Sara Jahedazad, Emma Lejeune·
arXiv:2610.38917v1 Announce Type: cross Abstract: Large language models (LLMs) have rapidly become a default source of help for students in science and engineering courses. In foundational engineering mechanics courses such as statics, students routinely use these tools to check …
arXiv:2610.39923v1 Announce Type: cross Abstract: As language models (LMs) have improved, an increasingly common means of evaluating their linguistic competence is through explicit prompting (e.g., "Which sentence is more acceptable?"). Prior work indicates that such prompting un…
arXiv cs.AI
TIER_1English(EN)·Mingzi Cao, Xi Wang, Nikolaos Aletras·
arXiv:2610.40091v1 Announce Type: cross Abstract: Effective multi-domain adaptation of large language models requires prioritizing relevant domain knowledge while minimizing interference from irrelevant domains. Existing approaches commonly use Low-Rank Adaptation (LoRA) to train…
arXiv cs.AI
TIER_1English(EN)·Jaden Park, Jungtaek Kim, Jongwon Jeong, Robert D. Nowak, Kangwook Lee, Yong Jae Lee·
arXiv:2604.13151v2 Announce Type: replace Abstract: Language Model (LM) agents are increasingly used in complex decision-making tasks, from AI coding to physical AI, where they must both explore a search space and exploit acquired knowledge effectively. Although exploration and e…
arXiv cs.AI
TIER_1English(EN)·Jinane Bazzi, Mariam Rakka, Fadi Kurdahi, Mohammed E. Fouda, Ahmed Eltawil·
arXiv:2604.11512v2 Announce Type: replace-cross Abstract: The growing demand for deploying Small Language Models (SLMs) on edge devices, including laptops, smartphones, automotive and robotic computing platforms, and high-performance embedded systems, has exposed fundamental inef…
Reading "While the man hunted the deer ran into the woods" requires giving up the analysis in which "the deer" is the object of "hunted". People who have just read such a sentence still produce more transitive structures, so the abandoned analysis primes later language. We ask wh…
arXiv cs.CV
TIER_1English(EN)·Samuel Dawit Assefa, Jae Won Cho·
arXiv:2610.38296v2 Announce Type: replace Abstract: Language-model answers are often treated as a visible record of the internal decision that produced them. We test this assumption in conflicting-context question answering, where two supplied documents support incompatible answe…
Medium — MLOps tag
TIER_1English(EN)·Vijay Dasari·
<blockquote> <p><strong>TL;DR</strong>: I built a language model <strong>from scratch, end to end, on free GPUs</strong>: a 125.9M-parameter model that writes in a register you choose, with slang, emoji, abbreviation, elongation and formality as five dials. The idea is simple and…
<!-- SC_OFF --><div class="md"><p>Cloud models have a very big problem which is very hard to solve, and it is not technological.</p> <p>The biggest issue with cloud providers is that nearly all tech companies face, but worse: they want to rip the benefits of language models, but …
