Beyond APIs: Probing the Limits of MLLMs in Physical Tool Use
Researchers have developed PhysTool-Bench, a new benchmark designed to evaluate how well Multimodal Large Language Models (MLLMs) can understand and use physical tools. The benchmark includes over 2,500 queries involving nearly 2,700 real-world tools across various industries. Testing revealed that even top-performing models struggle significantly, identifying only about 58.7% of tools and successfully completing just 21.0% of tasks, highlighting a critical gap in their ability to interact with the physical world. AI
IMPACT Highlights a significant limitation in current MLLMs for embodied AI, suggesting a bottleneck for real-world robotic applications.