BehaviorBench: Benchmarking Foundation Models for Behavioral Science Tasks

A new benchmark called BehaviorBench has been introduced to systematically evaluate how well foundation models perform on behavioral science tasks, according to a paper posted to arXiv on June 23 [1][2]. The benchmark assesses models across four core capabilities and reveals a performance gap between proprietary general-purpose models and those fine-tuned on behavioral data [2]. Foundation models are machine learning models trained on vast datasets so they can be applied across a wide range of use cases, with large language models being common examples [3]. Researchers have increasingly applied these models to behavioral science domains such as psychology, sociology, and economics [1][2]. However, the authors of the paper note that until now there has been no systematic understanding of how well the models perform across diverse behavioral science tasks, contexts, and populations [2]. BehaviorBench evaluates foundation models along four core capabilities: behavior prediction and simulation, strategic decision-making, subject-trait inference, and behavioral knowledge application [1][2]. The benchmark evaluates model outputs at both the individual and distributional levels, capturing per-subject accuracy as well as population-level alignment, which the researchers describe as an essential requirement for behavioral validity [2]. The paper's results reveal a considerable gap between model types. Proprietary general-purpose models excel at individual-level prediction and knowledge-intensive tasks, whereas behavioral foundation models fine-tuned on behavioral data achieve substantially stronger distributional alignment [1][2]. The researchers also developed Be.FM-1.5, extending the Be.FM family of behavioral foundation models [2]. Be.FM-1.5 leads on distributional metrics and remains competitive on individual-level metrics, suggesting that proper behavioral adaptation can close the performance gap [1][2]. Foundation models have been developed across a range of modalities beyond text, including images, music, and robotic control, as well as fields like astronomy and genomics [3]. The resource-intensive nature of building these models means adapting an existing foundation model for a specific task is typically far less costly than training one from scratch [3]. The BehaviorBench benchmark and Be.FM-1.5 models are accessible via a dedicated project page [2].