A new benchmark called Realtime-Bench has been developed by researchers from multiple universities and companies to evaluate the performance of AI systems in low-latency, real-time tasks. The benchmark includes 50 interactive web and game tasks designed to test temporal state awareness, adaptive deliberation, and time-aware action selection. Current frontier models, including those from OpenAI, Anthropic, and Google, performed poorly on the 'wall clock' evaluation, with the highest score being 36%, indicating significant limitations in their ability to handle time-sensitive operations. AI
IMPACT Highlights limitations in current AI models for real-time, low-latency tasks, suggesting areas for future development.
RANK_REASON The cluster describes a new benchmark and research paper evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Import AI (Jack Clark) →
- Anthropic
- Claude Fable 5-1
- Claude Opus-5
- Gemini 3.1-flash-live
- Gemini 3.8 Flash
- GPT 5.6 "Sol"
- GPT-Realtime-2
- Import AI
- Jack Clark
- Northwestern University
- Nvidia
- Ohio State University
- OpenAI
- OpenAI Decisions
- Realtime-Bench
- Steadyworks
- University of Illinois Urbana-Champaign
- University of Texas at Austin
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
