BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling
A new benchmark called BIM-Edit has been developed to evaluate the capabilities of large language models (LLMs) in editing Building Information Models (BIM) represented in the Industry Foundation Classes (IFC) format. The benchmark includes 324 editing tasks across 11 realistic and 36 synthetic building models, covering direct, spatial, and topological edits. Current LLMs show significant limitations, with the best-performing model achieving only a 49.5% score across geometric accuracy, semantic validity, and topological consistency, and failing to fully solve more than 3.4% of tasks. This highlights a substantial gap between LLM abilities and the demands of structured engineering design workflows. AI
IMPACT Highlights significant limitations in current LLMs for structured engineering design, indicating a need for further development in this domain.