Quaternion Self-Attention with Shared Scores

A new shared-score quaternion self-attention mechanism reduces computational cost in quaternion neural networks by cutting score-computation multiplications by 75% and consolidating four softmax operations into one, according to a paper posted to arXiv. [1][2] Quaternion neural networks encode four related features as a single entity, offering parameter efficiency and the ability to model multidimensional dependencies. [2] Standard quaternion self-attention, however, computes independent component-wise scores and applies separate softmax operations to each component, increasing computational load and allowing attention distributions to diverge. [1][2] The proposed method instead computes a single real-valued score using the quaternion inner product and applies a shared attention distribution across all components. [1][2] The authors prove that when queries and keys are produced by quaternion linear projections that induce component pre-mixing, the component-wise and shared scores lie in the same interaction subspace. [1][2] This finding indicates that independent component-wise attention primarily re-parameterizes the same interactions rather than expanding the feature interaction space. [1][2] In speech enhancement experiments, the shared-score method reduced inference time by up to 44.3% on a GPU and 58.1% on a CPU while maintaining output quality. [1][2] The paper reports consistent trends across vision and natural language processing tasks. [1][2] The work was submitted on 24 May 2026. [1]