DishwasherBench is a new game and benchmark designed to test the capabilities of frontier AI models in a simulated real-world task. Developed by Composo, the game involves loading a 3D model of a Bosch dishwasher with rigid-body physics, and then simulating the wash cycle to score how cleanly items are loaded. The benchmark is playable in a browser and also accessible via an API for AI agents, with initial tests involving six episodes across three levels using two models. AI
IMPACT This benchmark could reveal limitations in current AI models' ability to perform complex physical manipulation and spatial reasoning tasks.
RANK_REASON The item describes a new benchmark and simulation for evaluating AI models on a specific task.
Read on Mastodon — mastodon.social →
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
