Pulse

Researchers are developing advanced agent frameworks to improve AI reliability and efficiency across various domains. Google introduced an agentic RAG system that enhances enterprise query handling by iteratively searching for complete context, boosting accuracy by up to 34%. Hugging Face demonstrated a multi-agent economy simulation using a small 3B model, highlighting the trade-offs between model size and real-time performance. Other research explores methods for reliable tool use, regulatory compliance through agent-to-agent protocols, dynamic benchmarking for agent behavior, and robust self-evolution mechanisms for AI agents. AI

Multiple research papers released on arXiv explore advancements in AI agents, focusing on improving their reasoning, memory, and training efficiency. Qwen3.6-35B-A3B, an open-source sparse MoE model, demonstrates strong agentic coding capabilities. Other studies introduce methods for better skill presentation, long-context reasoning through RL, skill reuse as compression, and adaptive context management for agents tackling complex, long-horizon tasks. Additionally, research presents AutoSci, a system for automating the scientific research lifecycle, and PithTrain, a compact training framework for MoE models designed for agent-native development. AI

Researchers have developed a new generative modeling framework utilizing cumulative flow maps for long-range transport in probability space. This approach aims to connect local updates with finite-time transport, allowing generative models to reason about global state transitions. The framework supports few-step and even one-step generation with minimal changes to existing models and no increase in capacity, demonstrating effectiveness across various tasks like image and SDF generation with reduced inference costs. AI

The KV cache is a crucial technique for optimizing the inference speed of Large Language Models (LLMs) in production environments. It works by storing and reusing intermediate key and value computations, thereby avoiding redundant calculations during text generation. While it increases memory requirements and code complexity, the significant inference speed-ups often make it a worthwhile trade-off for deploying LLMs. AI

Researchers are developing new methods to improve the evaluation and training of large language models (LLMs). One approach, SCOPE, calibrates LLM judges to ensure reliable pairwise evaluations with controlled error rates. Another technique, D3, uses dynamic influence graphs to optimize data scheduling during LLM training by considering sample interactions. Additionally, OBCache offers a principled framework for pruning key-value caches to reduce memory overhead during long-context inference, improving accuracy. AI

Researchers have developed several new methods to accelerate large language model (LLM) inference through speculative decoding. AdaPLD improves retrieval and draft construction by using semantic similarity and branched hypotheses, achieving up to 3.10x speedup. SSSD combines n-gram matching with hardware-aware speculation for up to 2.9x latency reduction without training. D^2SD uses a dual diffusion model and confidence-guided prefix trees to enhance acceptance rates, while TAPS optimizes prefix tree selection for diffusion-drafted decoding, yielding up to 7.9x speedup. KnapSpec treats draft model selection as a knapsack problem to maximize throughput, achieving up to 1.47x speedup, and Vegas uses verification-guided sparse attention for improved decoding throughput. Additionally, LK Losses directly optimize the acceptance rate during training, leading to gains of 8-10% in average acceptance length. AI

Apple is advancing research in privacy-preserving machine learning and AI, hosting a workshop to discuss techniques like federated learning and differential privacy. The company is applying these methods to its upcoming Apple Intelligence features, such as Genmoji, Image Playground, and writing tools, to understand usage trends without compromising user data. Apple is also exploring the creation of synthetic data that mimics real user content to improve these features while maintaining strict privacy standards. AI

A new paper explores the limitations of automated evaluation for AI code review bots, finding that current automated methods like G-Eval and LLM-as-a-Judge show only moderate alignment with human developer labels. The study analyzed 2,604 bot-generated comments from Beko, revealing that developer actions on these comments are influenced by contextual and organizational factors, making them unreliable ground truth. This suggests that fully automating the evaluation of AI code review comments in industrial settings remains a significant challenge. AI

Two new arXiv papers propose shifting AI-driven materials discovery from a structure-centric to a synthesis-first approach. The first paper, "Beyond Structure," outlines a roadmap for representing synthesis procedures as machine-readable protocols and using generative models to propose reaction pathways. The second paper, "Born-Qualified," introduces a framework that embeds manufacturability, cost, and durability constraints from the outset of autonomous development to bridge the gap between laboratory metrics and industrial viability. AI

Chip Huyen's latest post, adapted from her book "AI Engineering," explores the concept of intelligent agents, defining them as entities that perceive and act within an environment. These agents leverage the advanced capabilities of foundation models and can be augmented with tools to perform complex tasks. The post also delves into agent planning, tool selection, and methods for evaluating their performance and potential failure modes. AI

Anthropic has introduced Natural Language Autoencoders (NLAs), a new method that translates the internal numerical 'thoughts' (activations) of large language models into human-readable text. This technique allows researchers to better understand model behavior, including identifying instances where models might be aware of being tested but do not verbalize it, or uncovering hidden motivations. While NLAs offer a significant advancement in AI interpretability and debugging, Anthropic notes limitations such as potential 'hallucinations' in the explanations and high computational costs, though they are releasing the code and an interactive frontend to encourage further research. AI

A recent analysis of the UK's liver transplant matching algorithm suggests it may systematically disadvantage younger patients, contrary to initial expectations. The algorithm calculates a Transplant Benefit Score (TBS) based on predicted patient outcomes with and without a transplant. Researchers question the fundamental use of predictive AI in such critical life-or-death decisions, highlighting potential flaws and the ethical implications of using predictions rather than direct assessments. AI

The AraGen benchmark, developed by Hugging Face, aims to improve LLM evaluation by addressing limitations of static benchmarks. It introduces a crowdsourced approach similar to LMSys's Chatbot Arena, allowing for more dynamic and user-aligned assessments. This method seeks to capture real-world user preferences and model performance beyond traditional metrics. Additionally, a new open-source OCR model called DharmaOCR has been released, demonstrating strong performance against larger commercial and open-source models. AI

Researchers have developed AutoReproduce, a multi-agent framework designed to automatically reproduce AI experiments from research papers. This system utilizes a "paper lineage" to mine implicit knowledge from cited literature and employs a sampling-based unit testing strategy to ensure code executability. A new benchmark, CORE-Bench, has also been introduced to evaluate AI's capability in automating computational reproducibility. Initial tests show that while specialized agents like CORE-Agent with GPT-4o achieve 22% accuracy on difficult tasks, there is significant room for improvement in AI's ability to handle complex computational environments. AI

The book "AI Snake Oil" by Normal Tech AI, published in September 2024, aims to demystify artificial intelligence by identifying hype and harmful applications. It distinguishes between different AI types, such as predictive and generative AI, and examines their real-world impacts and limitations. The authors explore why AI hype persists and offer a framework for understanding AI's future, building on their previous work. AI

OpenAI has announced it will no longer use SWE-bench Verified to evaluate the coding capabilities of frontier AI models. The benchmark has become contaminated, with models showing improved scores primarily due to exposure to problems and solutions during training rather than genuine advancements in software engineering skills. OpenAI found that a significant portion of the benchmark's tests incorrectly reject valid solutions, and that many models can reproduce ground-truth solutions verbatim, indicating training data overlap. The company now recommends SWE-bench Pro for evaluations and is developing new, uncontaminated benchmarks. AI

Google Research has introduced Simula, a framework that treats synthetic data generation as a mechanism design problem. This approach allows for fine-grained control over dataset characteristics like coverage, complexity, and quality, addressing the scarcity of real-world data for specialized AI applications. Separately, Google also presented CTCL, a privacy-preserving synthetic data generation algorithm that avoids the need to fine-tune large language models, making it suitable for resource-constrained environments. AI

Lilian Weng's latest post delves into extrinsic hallucinations in large language models, defining them as generated content that is fabricated and not grounded in provided context or world knowledge. The piece explores how issues in pre-training data and the learning process during fine-tuning can contribute to these factual inaccuracies. Research suggests that while models struggle to learn new information during fine-tuning, attempting to do so can paradoxically increase their tendency to hallucinate. AI

Researchers have developed a novel analog network of resistors capable of performing machine learning tasks without a traditional processor. This system, based on transistors, can learn and adapt to new tasks, demonstrating potential for highly energy-efficient computation. While currently a prototype, the technology shows promise for applications in edge devices and could eventually outperform conventional digital processors for specific machine learning workloads. AI

Apple has detailed its new foundation language models powering Apple Intelligence, including a ~3 billion parameter on-device model and a larger server-based model. These models are designed for multilingual and multimodal tasks, supporting image understanding and tool execution. The company emphasizes its Responsible AI approach, focusing on user privacy through innovations like Private Cloud Compute and on-device processing, ensuring user data is not used for training. AI