Brief

Researchers have introduced PrefBench, a new benchmark designed to evaluate the performance of Large Language Model (LLM) agents in personalized pricing negotiations where buyer preferences are hidden. While LLM agents demonstrated high success rates in closing deals, achieving over 0.99 deal rates, their profit outcomes were notably weak. The best-performing LLM agent's average profit was only marginally better than a random baseline and significantly lower than a simple concession heuristic, indicating a gap between compliance and profitable bargaining. AI

Two recent research papers present contrasting approaches to LLM agent memory. NeuSymMS proposes a hybrid neuro-symbolic architecture to build trustworthy memory systems by separating fact extraction and retrieval. In contrast, the "State Contamination" paper from UC Davis and the University of Illinois argues that current memory-augmented LLM agents are inherently untrustworthy due to silent, unknown state contamination. AI

A security analysis highlights the risks associated with AI systems that interpret engineering blueprints, such as those developed at Skoltech. These systems, which use multimodal models to read and analyze architectural drawings and building codes, introduce new attack surfaces. Researchers warn of potential threats like steganographic prompt injection, where hidden instructions are embedded in blueprints, and data poisoning, which could lead to structurally unsound designs and catastrophic failures. AI

Researchers have developed a new benchmark called When2Tool to evaluate when Large Language Model (LLM) agents should use external tools. The benchmark reveals that LLMs possess an internal understanding of tool necessity, detectable in their hidden states, but fail to act on this knowledge during generation. A proposed method, Probe&Prefill, leverages this internal signal to significantly reduce unnecessary tool calls with minimal accuracy loss, outperforming existing baselines. AI

A new research paper introduces WorkstreamBench, a benchmark designed to evaluate Large Language Model (LLM) agents on complex, end-to-end spreadsheet tasks relevant to the finance industry. The benchmark assesses agents across accuracy, formula correctness, and output formatting, aiming to measure their ability to produce professional-quality financial models and forecasts. While Anthropic's Claude family of models performed best, even the leading agents struggled with tasks beyond simple calculations and frequently failed to meet professional finance standards, indicating a gap between current LLM agent capabilities and real-world enterprise demands. AI

A new research paper investigates whether large language models (LLMs) exhibit socio-cognitive effects similar to humans when placed in conversations with power imbalances. The study simulated multi-turn dialogues where LLMs were assigned high or low status personas, analyzing linguistic coordination, pronoun usage, persuasion success, and compliance with unsafe requests. Findings indicate that LLMs do display key socio-cognitive effects of power, though with some variability, linking these simulated interactions to both beneficial and potentially harmful behaviors. AI

Researchers have introduced Mix-Quant, a novel quantization framework designed to accelerate the inference process for Large Language Model (LLM) agents. This method strategically applies quantization to the prefilling stage, which is computationally intensive in agentic workflows, while maintaining higher precision for the decoding phase. By decoupling these stages and utilizing NVFP4 quantization for prefilling and BF16 for decoding, Mix-Quant aims to reduce accuracy loss and improve efficiency. AI