Brief

Researchers have developed MDIA, a Multi-agent Diagnostic Intelligence Agent, which utilizes a 7-node clinical reasoning graph to achieve strong performance on the HealthBench Professional benchmark. When evaluated using OpenAI's GPT-5.4-2026-03-05, MDIA scored 0.6272, surpassing ChatGPT for Clinicians by 3.72 percentage points. The study indicates that architectural design, including specialty routing and context preservation, significantly impacts agentic performance, rather than solely prompt engineering. The choice of grading model also introduces variability, as demonstrated by MDIA scoring 0.6585 when graded by Gemini 2.5 Pro, highlighting the need for multi-grader evaluations. AI

A recent benchmark compared GPT-4.1, Claude Sonnet 4.5, and Gemini 2.5 Pro on real-world coding tasks. Claude Sonnet 4.5 scored highest in code generation, demonstrating strong structural consistency and appropriate use of advanced libraries like asyncio. Gemini 2.5 Pro excelled in complex reasoning tasks and provided the most detailed explanations, while GPT-4.1 handled ambiguity by asking clarifying questions, though it made reasonable assumptions when forced to produce output. AI

Two separate teams at Nexus Labs and Prophesee have adopted Bifrost, an open-source gateway, to manage their interactions with multiple large language models. Prophesee used Bifrost to caption 1.2 million robotics frames, achieving a 22% cost saving by intelligently routing requests across GPT-4o, Claude 3.7 Sonnet, and Gemini 2.5 Pro. Nexus Labs implemented Bifrost to improve the quality of their agent training data, finding that nearly half of their production traces were unusable due to inconsistent model behavior and hidden provider failures. By using Bifrost's advanced fallback and logging features, they were able to reduce corrupted traces from 17% to under 3%, enabling more reliable fine-tuning. AI

Researchers at CISPA audited 17 third-party "shadow" LLM APIs and discovered significant performance discrepancies compared to the official models they claimed to represent. These services often provide access to cheaper or entirely different models, leading to degraded accuracy in academic research. The study identified three common substitution patterns: silent downgrades, cross-vendor swaps, and partial routing based on context length, with simple fingerprinting tests capable of detecting many, but not all, of these deceptions. AI

A recent benchmark evaluated six large language models on their ability to extract structured data, specifically JSON, from customer support emails. The analysis found that Anthropic's Claude Haiku 4.5 offered the best value, achieving high accuracy at a significantly lower cost compared to more powerful models. While Gemini 2.5 Flash was fast and inexpensive, it struggled with accuracy, particularly in hallucinating data. The study suggests using Haiku for most extraction tasks, Sonnet for more complex reasoning, and avoiding more expensive frontier models for simple data extraction. AI

NemoStation has released Marlin-2B, a compact video large model (VLM) designed for extracting structured information from videos. This 2-billion parameter model excels at dense captioning and temporal grounding, outperforming other models in its weight class on benchmarks like CaReBench and TimeLens-Bench. Marlin-2B is optimized for deployment, capable of running on a single consumer GPU and offering developer-friendly APIs for easy integration into applications. AI