TensorBench: Benchmarking Coding Agents on a Compiler-Based Tensor Framework

Researchers have introduced two new benchmarks, TensorBench and SWE-Explore, to evaluate the capabilities of coding agents in repository-level coding tasks.

TensorBench, introduced in a paper on arXiv^[1], is a benchmark for coding agents on a compiler-based tensor framework that extends PyTorch. It consists of 199 feature-addition and refactoring tasks, covering new sparse formats, dense optimization passes, and IR transformations. Seven coding agents were evaluated on TensorBench, with pass rates ranging from 64.8% to 22.1%^[1]. The benchmark grades each run by applying the agent's patch and running the framework's test suite. In a separate development, researchers introduced SWE-Explore, a benchmark that evaluates the repository exploration capability of coding agents^[2]. SWE-Explore covers 848 issues across 10 programming languages and 203 open-source repositories, asking an explorer to return a ranked list of relevant code regions under a fixed line budget. The benchmark evaluates exploration along coverage, ranking, and context-efficiency dimensions. According to the researchers, agentic explorers form a clear tier above classical retrieval methods in terms of exploration capabilities^[2].