Variational Consensus Monte Carlo for Bayesian Mixture

Researchers have proposed new methods for inferring Bayesian mixture models in federated learning settings and approximating the LogSumExp function, a crucial component in many optimization problems.

The new method for Bayesian mixture models uses a Consensus Monte Carlo (CMC) approach to estimate local posterior distributions, which are then aggregated to approximate the posterior over the full data^[1]. This approach is particularly useful in cross-silo settings where not every cluster appears in each local dataset. The researchers extended the variational CMC approach to over-fitted Bayesian mixture models, allowing for the inference of the number of clusters and all model parameters without requiring conjugacy. A comprehensive simulation study validated the framework, showing that it can recover small clusters with greater accuracy than standard MCMC applied to the pooled data^[1]. Meanwhile, a separate research effort proposed a novel convexity- and smoothness-preserving approximation to the LogSumExp function, rooted in a modification of the KL divergence in the dual, resulting in a new $f$-divergence called the Safe KL divergence^[2].