A General Framework for Decision Trees via Bregman Divergences

Researchers have proposed a new framework for decision trees that generalizes the widely used Classification and Regression Trees (CART) algorithm by building it on Bregman divergences, a family of loss functions first introduced in 1967 [1, 2]. The framework, submitted for publication on 12 June 2026, aims to unify the way impurity criteria are selected for decision trees, a fundamental tool in statistical learning valued for interpretability and flexibility [1, 2]. The CART algorithm, introduced by Breiman, Friedman, Olshen, and Stone in 1984, remains one of the most influential methods for classification and regression [1, 2]. Its classical implementations, such as rpart, typically incorporate different impurity criteria in an ad hoc manner for each specific model [2]. The new work instead derives these criteria from common convex and geometric principles [1, 2]. Bregman divergences, introduced by Lev Bregman in 1967 in the context of convex optimization, provide a broad family of loss functions that generalize the squared Euclidean distance [1, 2]. This family includes the Kullback-Leibler divergence, the Poisson divergence, and the Itakura-Saito divergence, as well as losses associated with distributions in the exponential family [2]. The authors argue that the Bregman divergence approach yields a broader family of decision trees adapted to different statistical models and underlying geometries [1, 2]. Beyond the algorithmic construction, the paper investigates theoretical properties of these trees [1, 2]. It examines how properties of the generating convex function—such as strong convexity or smoothness—influence impurity gains between parent and child nodes, as well as the stability and consistency of the estimator [2].