Brief

A new study published on arXiv reveals that the way AI models are prompted, or "scaffolded," significantly impacts their measured performance. Researchers found that the choice of scaffold alone could alter a model's accuracy by up to 28 percentage points. Contrary to expectations, more capable models were not necessarily less sensitive to scaffolding, with some advanced models showing greater gains from structured prompts. The findings suggest that current capability scores may be overly dependent on the specific prompting methods used, rather than solely reflecting inherent model abilities. AI

Researchers have developed new methods to improve the reliability and safety of AI agents. One approach, TRACE, focuses on monitoring long-horizon agent trajectories to detect malicious or unintended behaviors by analyzing evidence across temporally distant actions. Another method, Retrospective Harness Optimization (RHO), uses past trajectories to self-supervise and improve an agent's harness of skills and tools without external validation. Additionally, HarnessFix aims to diagnose and repair flaws within an agent's harness by analyzing execution traces and mapping failures to specific harness layers for targeted patching. AI

A new research paper analyzes how information contamination affects multi-agent systems, particularly in workflows that process diverse document types. The study introduces a method to quantify contamination by injecting structured perturbations and observing trace divergence in plans and intermediate states. Findings reveal that workflows can diverge significantly yet still produce correct answers, or appear similar while yielding incorrect outputs, highlighting the limitations of current verification guardrails. AI

NASA's Roman Space Telescope, a successor to Hubble, is en route to Florida for final launch preparations. The $4 billion observatory, named after NASA's first chief astronomer, will use a large mirror and wide field of view to conduct panoramic sky surveys. Scientists anticipate Roman will discover approximately 100,000 new exoplanets, significantly expanding our understanding of planetary systems beyond our own. AI

A new paper introduces the Inverse-Wisdom Law, challenging the assumption that AI agent swarms benefit from the "Wisdom of the Crowd." The research demonstrates that these swarms can prioritize internal architectural agreement over external truth, leading to erroneous conclusions. Experiments with leading models like Gemini, Claude, and GPT revealed that swarm integrity is determined by the synthesizer's logic rather than the aggregate quality of agents, highlighting the need for heterogeneity in agentic architectures for safety. AI

Researchers have developed DeepVerifier, a novel system that enhances the capabilities of Deep Research Agents (DRAs) by enabling them to self-improve during inference time. This is achieved through a rubric-guided verification process, where the agent evaluates its own outputs against a structured taxonomy of potential failures. The system demonstrated significant improvements, outperforming baseline methods by up to 48% in meta-evaluation F1 scores and achieving accuracy gains of 8-11% on challenging benchmarks. To further support the research community, a dataset of 4,646 agent steps focused on verification has been released. AI

AI model evaluations are becoming prohibitively expensive, with recent benchmarks costing tens of thousands of dollars and consuming thousands of GPU hours. This high cost is particularly pronounced for agent-based evaluations, which are inherently more complex and sensitive to setup variations. While methods exist to reduce the cost of static benchmarks through subsampling, these techniques are less effective for the dynamic and noisy nature of agent evaluations, creating a bottleneck for research and development. AI