CADBench: A Multimodal Benchmark for AI-Assisted CAD Program Generation

A research team has released CADBench, a unified benchmark designed to measure how well AI systems generate editable CAD programs from 2D images and 3D observations, addressing what its creators describe as a fragmented evaluation landscape [1]. The benchmark assembles 18,000 evaluation samples drawn from six established dataset families: DeepCAD, Fusion 360, ABC, MCB, and Objaverse [1]. It supports five input modalities — clean meshes, noisy meshes, single-view renders, photorealistic renders, and multi-view renders — and evaluates outputs across six metrics that capture geometric fidelity, executability, and program compactness [2]. STEP-based families within the benchmark are stratified by B-rep face count, and all families undergo diversity sampling to enable controlled analysis across geometric complexity and object variation [3]. The researchers benchmarked eleven systems, spanning CAD-specialized models and general-purpose vision-language models, producing more than 1.4 million CAD programs during evaluation [1]. Under idealized input conditions, specialized mesh-to-CAD models substantially outperformed code-generating vision-language models, which the paper describes as remaining far from reliable CAD program reconstruction [4]. CADBench surfaces three recurring failure patterns. Reconstruction quality degrades as geometric complexity increases. CAD-specialized models prove brittle when the input modality shifts away from the clean conditions they were trained on. Model rankings also change depending on which metric is prioritized, underscoring the risk of relying on a single evaluation dimension [5]. The benchmark is publicly available on Hugging Face and GitHub [2]. The authors position CADBench as a diagnostic testbed for tracking progress in editable 3D reconstruction and multimodal CAD understanding [3].