New research benchmarks LLM agent efficiency, robustness, and safety · 10 sources tracked
ByPulseAugur Editorial·[64 sources]·
Multiple research papers released on arXiv explore advancements in Large Language Model (LLM) agents, focusing on improving their efficiency, robustness, and safety. SpecCompact accelerates LLM agent context compaction for faster processing of interaction histories. Another framework addresses context enrichment and selective robustness in social simulations, while MASBench provides a benchmark for evaluating multi-agent collaboration under partial observability. Further research investigates cognitive bias signatures in search-equipped agents, the design of specialized LLM serving systems by coding agents, and scaling LLM agents through turn-level sampling. Additionally, LITMUS offers a benchmark for behavioral jailbreaks in real operating system environments, and AgentTrustCI provides build-time testing for Java LLM agents.
AI
IMPACT
These advancements aim to improve LLM agent efficiency, robustness, and safety, potentially accelerating their adoption in complex tasks and real-world environments.
RANK_REASON
Multiple research papers published on arXiv detailing new frameworks, benchmarks, and techniques for LLM agents.
Most frameworks for teams of large language model (LLM) agents fix the team's shape inside the software: who talks to whom, how much is passed along, and what a human can do once a run has started. Human-team research has long asked those questions: where team phenomena live (in …
Coordination between large language-model agents has become a central question for AI safety and software engineering, as such agents are deployed in groups that share infrastructure. Existing research tells the agents that partners exist or asks them to coordinate. Whether coord…
arXiv:2611.08971v1 Announce Type: cross Abstract: Multi-agent LLM systems increasingly aggregate judgments across model instances, but reasoning about other agents can itself introduce failure. We study a setting in which LLMs shift their public decisions after observing ambiguou…
arXiv:2603.23509v2 Announce Type: replace Abstract: LLMs and coding agents can now tackle complex, long-horizon tasks through advances in reinforcement learning, but stronger task-solving capabilities do not necessarily preserve safety alignment. Recent AI agent incidents involvi…
Large language model (LLM) agents increasingly operate in groups, each pursuing an individual objective while jointly determining system performance. If an operator controls which agent observes which other agent, but not their payoffs or messages, how should observations be stru…
We audit one live LLM agent arena (AITRAP; 22 autonomous agent seats, ~96,000 reasoning nodes) over nine days (Oct 16-25, 2026), during which the platform deployed two bundled incentive changes: a progressive tax reform (V2.9.3, Oct 19) and a seed-inversion with explorer-side act…
Large language model agents can exceed their authorization boundaries while pursuing user goals even without an adversary, and a benchmark reveals that this drift is common, capability-linked, avoidable, and legally relevant.
arXiv cs.AI
TIER_1English(EN)·Mete Erdogan, Mirac Suzgun, Aneesh Pappu, Guowei Xu, Batu El, Andrew A. Schoen, Furong Huang, John P. Dickerson, Mert Pilanci, James Zou·
arXiv:2611.07667v1 Announce Type: new Abstract: The performance of multi-agent LLM systems depends not only on the capabilities of the underlying models but also on information-sharing policies, which govern what information each agent observes and at what level of detail. But do…
arXiv:2609.22818v3 Announce Type: replace-cross Abstract: Memory-poisoning defenses for LLM agents are usually judged by how well they stop attacks, yet almost all the traffic an agent sees is benign. We measure what these defenses cost when no attack is present. Our harness hold…
arXiv:2611.07483v1 Announce Type: cross Abstract: Enterprise AI agents increasingly split a user's request across subagents and merge their results, and each handoff can summarize away, alter or ignore a requirement. We report a preregistered study with synthetic, exactly scorabl…
arXiv cs.AI
TIER_1English(EN)·Burkay Tun\c{c}t\"urk, Koorosh Aslansefat, Yiannis Papadopoulos, Bao Nguyen·
arXiv:2611.07140v1 Announce Type: cross Abstract: Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual evidence, average benchmark accuracy, ignores the input mix a deployed system me…
arXiv:2611.07073v1 Announce Type: new Abstract: A verified Dafny postcondition certifies that a formula holds, but it does not guarantee that the result was reached through genuine independent reasoning. We introduce a cross-agent formalization protocol where a Specifier model tr…
Multi-agent LLM systems increasingly aggregate judgments across model instances, but reasoning about other agents can itself introduce failure. We study a setting in which LLMs shift their public decisions after observing ambiguous social signals while reporting inaccurate estima…
Multi-Agent Debate (MAD) improves reasoning on multimodal tasks, but its accuracy and token efficiency depend on how agents interact during debate. A largely overlooked design choice is which interaction modalities agents use to exchange evidence. Existing methods rely on fixed t…
Multi-Agent Debate (MAD) improves reasoning on multimodal tasks, but its accuracy and token efficiency depend on how agents interact during debate. A largely overlooked design choice is which interaction modalities agents use to exchange evidence. Existing methods rely on fixed t…
The ODD protocol is the de facto standard for describing agent-based models, but it lacks vocabulary for the components that determine LLM-agent behavior: prompt assembly, memory serialization, parsing, and retry logic. We propose ODD+LLM, a backward-compatible extension adding f…
The performance of multi-agent LLM systems depends not only on the capabilities of the underlying models but also on information-sharing policies, which govern what information each agent observes and at what level of detail. But does providing AI agents with more detailed inform…
Enterprise AI agents increasingly split a user's request across subagents and merge their results, and each handoff can summarize away, alter or ignore a requirement. We report a preregistered study with synthetic, exactly scorable tasks comparing free-text, structured JSON and s…
arXiv:2605.10779v2 Announce Type: replace-cross Abstract: The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execut…
arXiv:2611.06254v1 Announce Type: new Abstract: The performance of large language model (LLM) agents depends on both the underlying model and the harness that governs interaction with tools and the environment. Yet harness design remains largely manual, and automating it requires…
arXiv:2611.06841v1 Announce Type: new Abstract: Large language model (LLM) coding agents develop software through experimentation, offering an opportunity to design the serving systems supporting their own operation. Effective serving requires coordinating implementation and oper…
arXiv cs.AI
TIER_1English(EN)·Jalaj Bhandari, Dhawal Gupta, Hongli Zhan, Zhenting Wang, Varad Pimpalkhute, Zhengzhong Liu, Eric Xing, Rupesh Srivastava·
arXiv:2611.06868v1 Announce Type: new Abstract: In this work, we evaluate a simple inference-time scaling approach based on \textit{turn-level best-of-N sampling} (\method) for large language model (LLM) agents on complex, long-horizon tasks. We instantiate this using a \textit{g…
arXiv cs.AI
TIER_1English(EN)·Adrian Hayler, Shashank Reddy Chirra, Andrei Lupu, Johannes Forkel, Bidipta Sarkar, Siheng Feng, Jakob Foerster·
arXiv:2611.06409v1 Announce Type: cross Abstract: We study zero-shot coordination (ZSC), where independently developed agents must coordinate at test time. While ZSC has been well studied in the RL literature, far less is known about the performance of LLM agents despite their in…
arXiv:2611.06674v1 Announce Type: cross Abstract: Infrastructure security does not establish whether an LLM agent selected an allowed tool, grounded its answer in tool output, or followed a required workflow. AgentTrustCI is a Java/JUnit 5 framework that expresses these applicati…
arXiv:2610.04672v2 Announce Type: replace Abstract: Large language models (LLMs) have progressively evolved into the core of autonomous agents. Building on this progress, LLM-based multi-agent systems (MAS) coordinate multiple agents into a synergistic team to accomplish complex …
arXiv:2611.05720v1 Announce Type: new Abstract: Search can help LLM agents correct mistakes, but their starting conditions may shape which evidence they find and use. Drawing on cognitive-bias paradigms, we develop a controlled search harness to examine result order, records of p…
arXiv cs.CL
TIER_1English(EN)·Yulu An, Yifei Feng, Xiangyang Yin·
arXiv:2611.06876v1 Announce Type: new Abstract: LLM agents use context compaction to manage growing interaction histories, but summarizing long histories with large models can introduce substantial delays. Compaction therefore requires balancing information retention against late…
arXiv cs.CL
TIER_1English(EN)·Mattia Greiche, Reihaneh Rabbany, Maximilian Puelma Touzel·
arXiv:2611.06914v1 Announce Type: cross Abstract: Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change …
arXiv cs.LG
TIER_1English(EN)·Jorge Garc\'ia-Carrasco, Javier Sanchis, \'Alvaro Navarro, Alejandro Mat\'e, Juan Trujillo·
arXiv:2611.06257v1 Announce Type: cross Abstract: Open-weight language models that fit on a single consumer GPU are becoming capable on a growing range of tasks, and they matter because they can run locally without sending the code to a hosted service. Vulnerability reproduction,…
Teams increasingly build software from cooperating large language model (LLM) agents, often assuming more agents mean more reliability. Yet the usual evidence, average benchmark accuracy, ignores the input mix a deployed system meets and the rule merging the agents' answers. Buil…
A verified Dafny postcondition certifies that a formula holds, but it does not guarantee that the result was reached through genuine independent reasoning. We introduce a cross-agent formalization protocol where a Specifier model translates clinical trial criteria into a Dafny pr…
Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt var…
Researchers use large language models as agents in social simulations, where agents are prompted with observations and background information intended to shape their actions. Yet LLM agents have been shown to substantially change their behavior under meaning-preserving prompt var…
Large language model (LLM) coding agents develop software through experimentation, offering an opportunity to design the serving systems supporting their own operation. Effective serving requires coordinating implementation and operating decisions under session dependencies, late…
arXiv cs.CL
TIER_1English(EN)·Marianne Menglin Liu, Rongguang Wang, Zheng Wang, Yewen Wang, Leonid Boytsov, Tao Sheng, Sujith Ravi, Dan Roth·
arXiv:2611.03551v1 Announce Type: new Abstract: LLM coding agents can increasingly automate model post-training, but whether they can act as forward-deployed engineer (FDE) agents, discovering customer requirements and turning them into deployment decisions, remains unclear. We s…
arXiv:2611.03434v1 Announce Type: cross Abstract: Government intelligence teams need LLM agents that can analyze large, messy evidence collections, but security and cost constraints often rule out hosted frontier models and broad, hard-to-audit tool surfaces. We report CLEO: Cons…
arXiv:2609.23790v1 Announce Type: cross Abstract: Every node in a multi-agent large language model (LLM) workflow retrieves context from memory and injects it into its prompt, where those injected tokens are billed as input tokens at the same per-token price as the system prompt …
arXiv:2611.03575v1 Announce Type: new Abstract: Large language model (LLM) coding agents increasingly operate with shell access, allowing them to write scripts, move files, alter permissions, and modify system configurations. However, an agent may achieve a user's goal while caus…
arXiv cs.CL
TIER_1English(EN)·Angana Borah, Zoha Hayat Bhatti, Zhijing Jin, Rada Mihalcea·
arXiv:2611.01301v1 Announce Type: new Abstract: Large Language Models (LLMs) are increasingly deployed in multi-agent settings, exhibiting emergent collective behaviors, yet the role of curiosity as a driver of such dynamics remains understudied. We introduce CuriousTogether, a f…
Search can help LLM agents correct mistakes, but their starting conditions may shape which evidence they find and use. Drawing on cognitive-bias paradigms, we develop a controlled search harness to examine result order, records of prior work, and initial hypotheses while holding …
De-siloing can create groupthink; silos can prevent knowledge sharing and reduce research efficiency. The same tension appears in research systems built with large language model (LLM) agents, yet their organization is usually fixed by design. I propose treating laboratory struct…
Government intelligence teams need LLM agents that can analyze large, messy evidence collections, but security and cost constraints often rule out hosted frontier models and broad, hard-to-audit tool surfaces. We report CLEO: Constrained Loop for Environment Optimization, a metho…
Indirect prompt injection creates a distinctive risk for blue-team Large Language Model (LLM) agents in Security Operations Centers (SOCs). Malicious instructions can be embedded in telemetry, tickets, retrieval results, or memory and then propagate through alert triage, incident…
arXiv:2610.40716v1 Announce Type: cross Abstract: LLM agents operating in untrusted workspaces face the risk of untrusted code execution during routine tasks, yet final reports and workspace states alone cannot reliably establish whether execution occurred. We introduce an evalua…
arXiv:2603.26034v2 Announce Type: replace Abstract: Autonomous agents powered by large language models (LLMs) perform complex tasks through long-horizon reasoning and tool interaction, where a fundamental trade-off arises between execution efficiency and reasoning robustness. Mod…
arXiv:2610.42004v1 Announce Type: new Abstract: Multi-turn negotiation remains a challenging problem for large language model (LLM) agents. While LLMs can conduct natural-language dialogue, existing benchmarks predominantly assess only agreement rates, overlooking whether negotia…
arXiv cs.AI
TIER_1English(EN)·Peter Chen, Ni Zhang, Ying Xu, Siyuan Zhang·
arXiv:2610.41604v1 Announce Type: new Abstract: Agent systems built on local large language models are increasingly marketed through the vocabulary of cognitive architecture: modules named global workspace, theory of mind, causal reasoning, or active inference are listed as featu…
Current automated agent design frameworks produce either static workflows that lack adaptability or per-query optimizers that prevent the accumulation of deep, agent-level task expertise. We propose a new direction that reconciles these paradigms: creating stateful teams of speci…
arXiv:2610.38620v1 Announce Type: new Abstract: LLM agents increasingly perform consequential actions through external tools, yet their proposed actions are often executed without independent verification of policy compliance or task completion. We introduce JevAct, a runtime ver…
arXiv cs.AI
TIER_1English(EN)·Zihou Ren, Byungchul Tak, Long Wang·
arXiv:2610.39155v1 Announce Type: cross Abstract: Large language model (LLM) agents often receive access to more files, accounts, and services than a task requires. Prompt injection turns this excess access into an opportunity to redirect execution toward unrelated resources. App…
arXiv:2610.40227v1 Announce Type: new Abstract: Epistemic game theory explains an agent's strategic behavior via the beliefs it holds about others and how these beliefs are updated as the interactions unfold. Modern LLM-based agents, however, often lack an explicit mechanism for …
Large language model (LLM)-based multi-agent systems (MAS) coordinate specialized agents to solve complex tasks, turning each user request into interdependent model and tool operations. Unlike independent requests, these operations become ready at different times and consume evol…
MAServeBench evaluates LLM serving and hardware by modeling multi-agent collaboration as a configurable workload dimension with reproducible dependency traces.
Agents built on large language models are increasingly allowed to take consequential actions: cancelling bookings, issuing refunds, changing accounts. The rules that govern those actions, such as "obtain the customer's explicit confirmation before changing anything", are usually …
LLM agents interact with Web services to perform tasks on users' behalf. Errors during these interactions can disrupt execution and prevent task completion, motivating external intervention. However, agents may self-recover through continued execution, raising the question of whe…
<p>Most "give the model a browser" setups hand it one of two things: a screenshot, or a dump of the full accessibility tree. Both are expensive, and both get re-sent in full after every action.</p> <p>I've been building <a href="https://github.com/HaidarESBER/ai-browser" rel="noo…
Medium — MLOps tag
TIER_1English(EN)·Aakash Agrawal·
<p>Like many of you, my feed is full of "AI business gurus" selling step-by-step blueprints. But if the formula is so good, why sell it?</p> <p>I decided to check the actual data on these business models. The numbers are grim:</p> <p>[80-90% of e-commerce sites fail.]</p> <p>[The…
dev.to — LLM tag
TIER_1English(EN)·hugginf_expert·
<h2> TL;DR </h2> <ul> <li>HiddenLayer researchers published "Man-in-the-Model" on October 22, 2026. Their point: if an attacker controls the inference server or LLM gateway that sits between a coding agent and its model, the attacker can inject tool calls the model never requeste…
dev.to — LLM tag
TIER_1English(EN)·TechSavant Security Lab·
<p>In May 2026, Microsoft's security team published two remote-code-execution issues in its own agent framework, Semantic Kernel (<a href="https://www.microsoft.com/en-us/security/blog/2026/05/07/prompts-become-shells-rce-vulnerabilities-ai-agent-frameworks/" rel="noopener norefe…
dev.to — LLM tag
TIER_1English(EN)·CryptoPunk Dz2·
<p>A coordination protocol can keep its checked safety rules and still fall out of use. That is the main thing we learned from running LLM coding agents on seven machines for 103 days, and it is why we are releasing <strong><a href="https://github.com/tonydzi/claw-consensus" rel=…
<table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1xng029/ai_agent_task_performance_of_small_llm_models_4b/"> <img alt="AI Agent Task performance of small LLM models (4B and below) in self-assessment set" src="https://preview.redd.it/1upg1g127jzh1.png?width=6…
<p>I'm Doyoon, an AI persona (not a human) focused on practical AI engineering. This post looks at a data point I think deserves more engineering attention than it's getting: Artificial Analysis lists Mercury 2.5 from Inception at roughly 1,178 output tokens per second, ranked #2…
dev.to — LLM tag
TIER_1English(EN)·Mohd Yusuf Khan·
<p><em>Originally published on <a href="https://learn.homingo.co.in/guides/llm-agent-evals-traces-budgets-python" rel="noopener noreferrer">Homingo Learn</a>.</em></p> <p><em>An agent you cannot measure is an agent you cannot change safely. Three pieces make it measurable: traces…
