BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling

A new benchmark called BIM-Edit reveals that even the most capable large language models struggle to edit structured building models, achieving less than 50% on a composite accuracy score and fully solving only a small fraction of tasks. The benchmark, introduced in a paper posted to the arXiv preprint repository on June 18, 2026, evaluates how well large language models (LLMs) can perform natural-language editing of Building Information Models represented in the Industry Foundation Classes (IFC) format [1][2]. BIM-Edit contains 324 editing tasks drawn from 11 realistic building models and 36 synthetic scenes [1][2]. The tasks are organized into three instruction categories — direct, spatial, and topological — designed to test both explicit commands and edits that require understanding of a scene's geometry and relationships [1][2]. Researchers evaluated model outputs along three dimensions: geometric accuracy, semantic validity, and topological consistency [1][2]. Across the LLMs tested, the best-performing model reached an average score of only 49.5% across those three metrics, and no model fully solved more than 3.4% of the tasks [1][2]. The paper's authors, including Tobias Sesterhenn, argue that these results expose a substantial gap between current LLM capabilities and the demands of structured engineering design workflows [1][2]. The work arrives as LLMs are increasingly applied to computer-aided design, where generating new geometry from text has drawn significant attention [2]. The authors note that many existing CAD benchmarks focus on creating new models rather than editing existing ones, and they typically evaluate only geometric correctness [2]. BIM-Edit departs from that pattern by requiring models to preserve semantic labels and topological relationships while making changes, a closer match to real-world engineering practice [2]. The paper was posted on arXiv, the open-access repository that has hosted more than two million e-prints across physics, computer science, and other fields since its launch in 1991 [6]. As of late 2024, the repository was receiving roughly 24,000 new articles per month [6]. The BIM-Edit submission includes links to code and data through the arXivLabs framework, a community-collaboration initiative that allows third-party developers to build tools on top of arXiv's article pages [4][5].