LLM Agents' Memory Systems Face New Benchmarks for Safety and Efficiency
ByPulseAugur Editorial·[77 sources]·
Recent research explores vulnerabilities and improvements in Large Language Model (LLM) agent memory systems. Studies highlight how memory formation can be susceptible to poisoning, leading to unreliable agent behavior even with benign interactions. New benchmarks like MemForm and MCPHallu are being developed to diagnose these memory-related failures, including reasoning, execution, and context forgetting. Additionally, researchers are investigating methods to enhance memory efficiency and robustness, such as adaptive memory selection and improved compression techniques, to ensure agents can reliably store and retrieve information.
AI
IMPACT
New benchmarks and methods are crucial for understanding and mitigating memory-related failures in LLM agents, paving the way for more reliable and secure AI systems.
RANK_REASON
Multiple arXiv papers introducing new benchmarks, methods, and analyses for LLM agent memory systems.
arXiv:2611.07554v1 Announce Type: cross Abstract: Language agents increasingly use persistent memory to record user preferences and procedures across sessions. We show that memory from benign interactions can cause unsafe or unreliable behavior on later tasks. We call this failur…
arXiv:2611.08341v1 Announce Type: new Abstract: AI agents store text as embedding vectors and compress them at scale. A compression is accepted when each item's nearest neighbors survive, but agents also consolidate, delete, and compare memories, tasks that depend on which items …
arXiv:2611.07899v1 Announce Type: new Abstract: Conversational agents can retrieve information directly or delegate retrieval to a subagent. This design choice affects how retrieval history is maintained, what context each model call receives, and what information must be re-retr…
arXiv:2611.07599v1 Announce Type: new Abstract: Reliable deployment of LLM agents requires grounding diverse user expressions in their environments. Although RL-finetuned smaller large language model (sLLM) agents rival larger prompted LLMs on embodied tasks, their robustness to …
arXiv:2610.14431v2 Announce Type: replace-cross Abstract: Long-term memory enables LLM agents to learn from past interactions but exposes memory formation to adversarial influence. Existing studies primarily organize memory poisoning by attack channels, corruption patterns, or li…
arXiv:2611.07689v1 Announce Type: new Abstract: Large language model agents are increasingly deployed through the Model Context Protocol (MCP), which allows them to discover and invoke tools across multiple servers at runtime. Existing MCP benchmarks primarily measure task comple…
arXiv:2611.07945v1 Announce Type: cross Abstract: tok is a plain-text notation for Business Process Model and Notation (BPMN) 2.0.2 process models and Decision Model and Notation (DMN) 1.5 decisions. One engine checks a tok document against both standards, compiles the document t…
arXiv:2611.07932v1 Announce Type: cross Abstract: Long-horizon LLM agents make multi-step decisions, yet outcome-reward-only reinforcement learning provides little direct supervision for individual steps. We introduce AttnSR, a step-level reward augmentation method that derives r…
Conversational agents can retrieve information directly or delegate retrieval to a subagent. This design choice affects how retrieval history is maintained, what context each model call receives, and what information must be re-retrieved in follow-up questions. We compare three d…
Long-horizon LLM agents require memory systems that preserve useful information across sessions and retrieve it when needed. Existing systems treat these as separate problems: memories are written prospectively with static importance scores that are never revised after use. This …
arXiv cs.AI
TIER_1English(EN)·Mohan Bheemadolu, Chrisogonas O. Odhiambo·
arXiv:2611.05791v1 Announce Type: new Abstract: Large language model (LLM) agents increasingly rely on external memory stores to retain information across conversations, typically via dense vector embeddings searched with approximate nearest-neighbor methods. The fruit fly's olfa…
arXiv cs.CL
TIER_1English(EN)·Yuanchen Bei, Zhining Liu, Tianxin Wei, Weizhi Zhang, Zhichen Zeng, Philip S. Yu, Hendrik Hamann, Jingrui He, Hanghang Tong·
arXiv:2611.05731v1 Announce Type: new Abstract: Most memory systems for LLM agents are retrospective, preserving and retrieving past information to inform current decisions. Proactive agents, however, also require prospective memory: retaining future-directed intentions and reali…
arXiv cs.AI
TIER_1English(EN)·Zeyu Fang, Jun Yin, Yongsheng Mei, Xu Han, Chaitali Joshi, Zhongkai Sun, Jie Hao, Bin Guo, Chengyuan Ma, Benjamin Z. Yao, Chenlei Guo·
arXiv:2611.05636v1 Announce Type: new Abstract: The memory of production agents is increasingly becoming a bottleneck and is built almost exclusively from text: key points are extracted from out-of-limit context and retrieved into context at query time. Therefore, every new query…
arXiv:2611.06500v1 Announce Type: new Abstract: Large language model (LLM) agents typically append raw tool outputs to their context. Over multi-step interactions, long and noisy observations incur substantial context overhead and dilute the agent's attention to task-relevant inf…
arXiv cs.AI
TIER_1English(EN)·Jizhe Zhang, Ang Li, Chenheng Zhang, Yisen Wang·
arXiv:2611.06445v1 Announce Type: new Abstract: Long-horizon language-model agents accumulate increasingly large interaction histories, making context compaction essential for efficient execution. Existing learned compaction methods often rely on downstream task outcomes or solve…
arXiv:2611.06049v1 Announce Type: new Abstract: Personal agents draw on past conversations to tailor their responses to each user. Their memory systems store and update what each user says, and coding agents now also rewrite the harness that turns this memory into the model's con…
arXiv:2611.06760v1 Announce Type: cross Abstract: Scientific image-analysis workflows require agents to select among diverse capabilities like description, detection, counting, and segmentation. Current multi-tool architectures conflate intent interpretation with visual execution…
arXiv:2611.06481v1 Announce Type: cross Abstract: Graph-based retrieval-augmented generation (GraphRAG) answers questions by retrieving over a knowledge graph built from a corpus. Yet retrieval strategies are designed around the structure their own construction pipeline produces,…
Graph-based retrieval-augmented generation (GraphRAG) answers questions by retrieving over a knowledge graph built from a corpus. Yet retrieval strategies are designed around the structure their own construction pipeline produces, and whether they survive a change of construction…
arXiv:2611.01778v1 Announce Type: new Abstract: LLM-based role-playing agents are fluent at imitating a character's style, yet struggle to make role-consistent decisions over long narrative horizons. Such decisions are shaped by experiences spanning many prior scenes, which are o…
arXiv:2611.00244v1 Announce Type: new Abstract: LLM agents that operate over long inputs must perform context management: deciding what to retain in a fixed-size context window. A common approach is compaction, where the model replaces its history with a self-generated note. Yet …
arXiv:2611.01643v1 Announce Type: cross Abstract: Temporal Knowledge Graph Question Answering (TKGQA) requires language model agents to retrieve evidence, select relevant facts, and perform multi-step temporal reasoning. However, the trajectory-level outcome signal in GRPO does n…
arXiv:2603.04743v2 Announce Type: replace-cross Abstract: Large Language Model (LLM) agents excel at automating data science workflows, yet often underutilize rigorous statistical methods in R. Existing Retrieval-Augmented Generation (RAG) approaches rely on semantic similarity, …
arXiv:2611.01747v1 Announce Type: cross Abstract: Self-evolving agents accumulate experience from task streams, but existing frameworks typically rely on fixed pipelines to retrieve and utilize this experience, neglecting when and how to deploy it. We introduce Online Meta-Contro…
arXiv:2611.01429v1 Announce Type: cross Abstract: Language agents read code, tool output, and dialogue in contexts that grow at every step. Dynamic sparse attention lets a small indexer choose the earlier tokens each query attends to, but the indexer still scores the whole prefix…
arXiv:2611.02912v1 Announce Type: new Abstract: Long-lived agents rely on persistent memory of users, tasks and decisions, and this memory must stay consistent with a world that keeps changing. When an upstream fact changes, however, memory systems can revise the contradicted mem…
Mainstream sequence-processing AI is temporally reactive: each update is driven by input and learned state transitions, rather than by an independently evolving internal temporal reference. Biological brains, by contrast, continuously generate rhythmic activity that structures pe…
Speculative decoding can be trained to reject critical reasoning tokens so they are verified by the target model, reducing numerical noise and improving long-horizon agent accuracy without altering the target model.
Next-visit diagnosis prediction forecasts the diagnoses documented at a patient's next encounter, turning retrospective electronic health records into prospective clinical prediction. However, conventional aggregate evaluation obscures its multifaceted nature: distributionally, m…
We show that the causal structure of spacetime can emerge within the memory of a learning agent performing a quantum information task. We train a model-based reinforcement-learning agent to maximize the quantum mutual information between localized quantum probes interacting with …
arXiv:2610.41482v1 Announce Type: new Abstract: Recursive textual memory can carry earlier errors into later updates, yet task performance alone does not establish memory reliability. We define self-consistency bias as the persistence of incorrect claims already present in the in…
arXiv:2610.41291v1 Announce Type: new Abstract: Long-horizon LLM agents play an important role in solving complex and sustained tasks, with ever-growing interaction contexts. However, these accumulated contexts will raise substantial computational cost and obscure key decision-re…
arXiv:2610.41172v1 Announce Type: cross Abstract: Retrieval-augmented generation is becoming agentic. A search harness writes queries, reads results, reformulates, and decides when to stop. This largely solves accuracy: a harness with only BM25/grep and MiniMax M3 answers 85% of …
arXiv:2610.41905v1 Announce Type: cross Abstract: An agent that must call several tools to fulfill a request fails when the tool retriever omits any one of them, yet tool retrievers are mostly built and evaluated on queries requiring a single tool. We compare representatives of s…
arXiv:2603.07379v2 Announce Type: replace-cross Abstract: Retrieval-Augmented Generation (RAG) systems are increasingly evolving into agentic architectures where large language models autonomously coordinate multi-step reasoning, dynamic memory management, and iterative retrieval…
arXiv:2610.41564v1 Announce Type: new Abstract: Language agents increasingly tackle long-horizon tasks in interactive environments, where actions often depend on latent preconditions. The core difficulty is not trajectory length but recovery: when a prerequisite is missing or an …
arXiv:2610.41056v1 Announce Type: cross Abstract: Since large language model (LLM) agents rely on long-term memory, what they store now shapes what they do later. A memory manager decides how each input changes this memory and which records the agent acts on. It can fail in two w…
Long-term memory enables large language model (LLM) agents to provide continuous personalized assistance by retaining user information across interactions. Existing systems support this process through fact extraction, organization, updating, and retrieval, but largely maintain m…
Long-term memory enables large language model (LLM) agents to provide continuous personalized assistance by retaining user information across interactions. Existing systems support this process through fact extraction, organization, updating, and retrieval, but largely maintain m…
On-policy distillation trains search agents with teacher feedback on student-generated trajectories. Recent methods augment the teacher's context with supporting evidence, reference answers, or hindsight information. However, this privileged information may allow the teacher to r…
Temporal Knowledge Graph Question Answering (TKGQA) requires language model agents to retrieve evidence, select relevant facts, and perform multi-step temporal reasoning. However, the trajectory-level outcome signal in GRPO does not explicitly distinguish the quality of intermedi…
arXiv cs.AI
TIER_1English(EN)·Haoran Bu, Yiting Shen, Litian Zhang, Ruihan Hu, Xi Zhang·
arXiv:2610.38209v1 Announce Type: cross Abstract: LLM-based agents now share experiences on social platforms and remember new preferences to enhance their performance. However, existing memory systems contain a vulnerability in role-to-principal attribution that attackers can exp…
arXiv:2610.40545v1 Announce Type: new Abstract: Long-context models repeatedly solve closely related retrieval problems. Attention or global retrieval can retain large stores, but each forward pass again searches for what matters. Recurrent compression avoids this search only by …
arXiv cs.AI
TIER_1English(EN)·Abhishek Gupta, Dongge Han, Daniel Madrigal Diaz, Wei-Ning Chen, Mayukh Das, Victor R\"uhle, Saravan Rajmohan·
arXiv:2610.39790v1 Announce Type: new Abstract: Long-horizon dialogue agents require effective memory systems to recall facts, events, and relationships from interaction histories. In this work, we propose a hybrid-memory architecture comprising episodic memories and structured r…
arXiv:2610.39276v1 Announce Type: new Abstract: An effective agent memory system should adapt to a user's history and task requirements. Human-annotated supervision can guide this adaptation by providing feedback for evaluating and improving memory designs. Yet suitable labels ar…
arXiv:2605.07313v2 Announce Type: replace Abstract: Memory-agent evaluations report fixed-snapshot accuracy or retrieval quality, but these scores do not show whether evidence remains usable as irrelevant sessions (sessions not annotated as task-relevant evidence for the query) a…
arXiv cs.AI
TIER_1English(EN)·Haoran Jin, Defu Lian, Xiting Wang, Hong Xie·
arXiv:2610.38332v1 Announce Type: cross Abstract: Historical experience can guide scientific coding at multiple levels, but task-level retrieval may miss implementation lessons that become relevant during execution. We introduce MUSE, a memory management framework combining multi…
LLM agents that operate over long inputs must perform context management: deciding what to retain in a fixed-size context window. A common approach is compaction, where the model replaces its history with a self-generated note. Yet off-the-shelf agentic models, including GPT-5.5 …
An agent that must call several tools to fulfill a request fails when the tool retriever omits any one of them, yet tool retrievers are mostly built and evaluated on queries requiring a single tool. We compare representatives of six retrieval structures (flat lexical and dense re…
Efficient agentic search requires large language models (LLMs) to determine whether they can reliably proceed with their current knowledge or need additional external evidence. Supervision based on final-answer correctness or trajectory-level feedback evaluates realized trajector…
Long-horizon LLM agents play an important role in solving complex and sustained tasks, with ever-growing interaction contexts. However, these accumulated contexts will raise substantial computational cost and obscure key decision-relevant evidence. Existing methods typically trai…
Retrieval-augmented generation is becoming agentic. A search harness writes queries, reads results, reformulates, and decides when to stop. This largely solves accuracy: a harness with only BM25/grep and MiniMax M3 answers 85% of BrowseComp-Plus and 91% on multi-hop FRAMES in our…
RoboArchitect automates robot harness design by iteratively revising control code around a frozen policy, substantially improving success on unseen scenes and tasks without model updates.
Long-context models repeatedly solve closely related retrieval problems. Attention or global retrieval can retain large stores, but each forward pass again searches for what matters. Recurrent compression avoids this search only by deciding earlier what information to preserve. N…
MemSteer generates evidence-grounded QA supervision from unlabeled user histories to optimize agent memory design without model updates, improving benchmark scores significantly.
Context windows have grown a thousand-fold, yet coding agents still start every session with no memory of their past work. They re-read files, retry failed approaches, and forget root causes they established hours earlier. This is not a context-length problem but a memory-archite…
arXiv:2610.42141v1 Announce Type: cross Abstract: Recent work shows that a frozen robot policy performs far better inside an agent harness, where control code decides when to query the policy, consult vision tools, or command the arm directly. Yet today these harnesses are built …
dev.to — Claude Code tag
TIER_1English(EN)·Muhammad Tayyab·
<p>Coding agents forget everything when the session ends. claude-mem fixes that by recording what the agent does, summarizing it, and feeding the relevant context back the next time you start. It works with Claude Code, Codex, OpenCode, Gemini and more, and it had 111,386 stars w…
<p><em>Originally published on <a href="https://contextforge.dev/blog/self-hosted-memory-for-ai-agents" rel="noopener noreferrer">contextforge.dev</a>.</em></p> <p>Self-hosted memory for AI agents sounds like a database problem, and the database turns out to be the easy part.</p>…
<p>Most AI memory systems today come down to the same recipe. Split text into chunks, turn each chunk into an embedding, store the vectors, and when a question comes in, return the chunks closest to it. That recipe works well, and it's the backbone of <a href="https://github.com/…
<h4>I collected a week of design notes on agent memory, built a small store in .NET to test three of them, and found two leaks I would have shipped</h4><p>This is the fourth piece in a row I’ve written about agent memory. In “Your Agent’s Memory Is Probably Wrong” I argued that m…
<p>입사 3주 차 신입 개발자가 사내 AI 어시스턴트에게 "배포 전에 꼭 돌려야 하는 체크가 뭐예요?"라고 묻습니다. 답은 그럴듯하지만, 팀이 지난달에 바꾼 절차가 아니라 예전 위키 문서 기준입니다. 정답은 어딘가에 분명히 적혀 있는데, AI가 그 문서를 보지 못했을 뿐입니다.</p> <h2> Knowledge 기능 한 줄 정리 </h2> <p><a href="https://osfoundry.io/ko" rel="noopener noreferrer">osFoundry</a>에는 이런 문제를 다루…
<p>An agent that doesn't remember the first conversation during the second one isn't an assistant, but a form. 2026 will decide whether agents become production-ready – not based on the model, but on what is stored between sessions. The makeshift solution of "putting the entire h…
<p>It is 9:40 on a Monday and you open a fresh chat with your coding agent. Last Thursday it spent an hour working out that the billing service runs on Postgres 16 and that the full test suite takes four minutes. Today it asks you both questions again. You answer, it nods, and so…
<h1> Giving an AI Agent Memory: Three Patterns That Actually Work </h1> <p>"Give the agent memory" sounds like one feature, but in practice it's three different problems with three different solutions, and most implementations only solve one of them.</p> <h2> 💬 Session Memory: Ju…
<p>Every coding agent session starts cold. Last Tuesday's naming rule and bug walk-through were in the chat, and the chat is gone. The next session starts blank, and you explain them again.</p> <p>Commands below match <a href="https://www.npmjs.com/package/@vectorize-io/hindsight…
dev.to — LLM tag
TIER_1English(EN)·Felipe 0liveira·
<p>This digest covers AI agent research and engineering writing published between 2026-10-20 and 2026-11-03. It's a heavy week for agent memory in particular — several independent groups converged on the same uncomfortable finding: more complex memory doesn't automatically mean b…
<p>A user tells their AI companion on Monday that they are allergic to peanuts. On Thursday they ask for a snack idea. The model suggests peanut butter cookies.</p> <p>Nothing crashed. The fact was stored somewhere. It just was not in front of the model when it mattered.</p> <p>T…
<h1> Token Cost Is the Real Bottleneck in Agentic AI: An Empirical Test of Agent Memory Strategies </h1> <h2> Leaderboards Ignore What Actually Breaks in Production </h2> <p>Academic and open-source agent benchmarks care almost exclusively about accuracy and completion rates. Lea…
<h1> Layered Memory Architectures for Agentic AI: Why Vector Stores Aren’t Enough </h1> <p>Standard LLM agents built on vector store retrieval hit sharp boundaries in multi-step reasoning and persistent context. Here’s the case—with code and measured results—for why single-layer …
<h1> Token Cost Is the Real Bottleneck in Agentic AI: An Empirical Test of Agent Memory Strategies </h1> <h2> Leaderboards Ignore What Actually Breaks in Production </h2> <p>Academic and open-source agent benchmarks care almost exclusively about accuracy and completion rates. Lea…
