LabOSBench: Benchmarking Computer Use Agents for Scientific Instrument Control
Researchers have introduced LabOSBench, a new benchmark designed to evaluate computer-use agents in scientific instrument control. This benchmark utilizes web-based simulators to overcome the practical challenges of testing agents on physical instruments, such as cost and safety risks. LabOSBench includes 96 subtasks across eight instrument simulators, covering a range of scientific workflows. Initial experiments show that while current agents can handle structured tasks, they struggle with feedback-driven operations and long-horizon execution. AI
IMPACT This benchmark could accelerate the development of AI agents capable of complex, real-world scientific tasks.