Any-Dimensional Invariant Universality

A new study proposes a systematic framework for proving universality in machine learning models that accept inputs of varying sizes, such as graphs and point clouds, a class of architectures whose theoretical properties have remained poorly understood [1][2]. Traditional universality proofs, which show that a model can approximate any function within a given class, are built for models with fixed-size inputs defined on compact domains [2]. Models that handle any-dimensional inputs—sequences of functions on growing-sized data—do not fit this framework, leaving their approximation capabilities largely uncharacterized [1][2]. The concept of universality itself has roots in statistical mechanics, where it describes properties of large classes of systems that are independent of their microscopic details, an idea formalized by Leo Kadanoff in the 1960s [4]. The authors resolve the mismatch by identifying any-dimensional functions with a single function on an infinite-dimensional limit space that contains inputs of all finite sizes and their limits [1][2]. They then leverage the symmetries of these inputs and the relations between inputs of different sizes to endow this limit space with a natural topology [2]. This topology yields rich families of compact sets, which are the standard setting for establishing universality [2]. The mathematical machinery of continuous symmetries is often described by Lie groups, which provide a natural model for transformations that can be smoothly composed and inverted [3]. Applying their approach, the researchers demonstrate that several existing architectures fail to achieve universality [1][2]. They also propose simple architectural modifications that restore the property [1][2]. The work does not include empirical benchmarks but focuses on the theoretical conditions under which any-dimensional models can be considered universal approximators. The framework relies on identifying invariants—properties that remain unchanged under transformations—a concept related to Casimir elements, which are distinguished operators in the center of a Lie algebra's universal enveloping algebra that commute with all elements of the algebra [5].