Geometric Second-Order Feature Correlation Learning for Self-Supervised Speech Emotion Recognition

A new method for aggregating features in self-supervised speech emotion recognition proposes a Second-Order Correlation layer that models feature relationships as covariance descriptors, moving beyond conventional first-order pooling that assumes feature independence [1]. The technique, detailed in a paper submitted to arXiv on June 4, 2026, targets a known bottleneck in speech emotion recognition (SER) [1]. Self-supervised learning (SSL) produces rich, context-heavy representations, but standard aggregation methods treat each feature in isolation, discarding higher-order relationships and ignoring the underlying Riemannian geometry of the data [2]. The proposed Second-Order Correlation (SOC) layer instead captures synergistic co-occurrence patterns among features, forming discriminative signatures for identifying emotions [2]. To make these covariance descriptors usable in standard learning frameworks, the method applies a Log-Euclidean mapping (LEM) to project them from a Riemannian manifold into a Euclidean tangent space [2]. This step preserves geometric integrity while enabling direct linear discriminative learning [1]. The approach was tested on the ESD and RAVDESS datasets, where it recovered discriminative information lost during first-order pooling and effectively aggregated high-dimensional SSL features [2]. Neural networks, the broader family of models this work belongs to, consist of connected layers of artificial neurons that process data through weighted connections adjusted during training [4]. Deep learning architectures, which use multiple hidden layers, have been applied across computer vision, speech recognition, and natural language processing [3]. The arXiv preprint server, where the paper appears, hosts over two million articles and receives roughly 24,000 submissions per month as of late 2024, though papers there are moderated but not peer-reviewed [6]. While the SOC layer focuses on feature aggregation within a single model, other machine learning strategies such as ensemble methods combine multiple distinct models to improve predictive performance [5]. The SOC approach instead operates inside a single architecture, reformulating how internal representations are pooled before a final classification decision [2].