Topological Neural Dynamics: A Neuron-wise Framework for Sequence Modeling

A new sequence-modeling framework called Topological Neural Dynamics (TND) has been introduced, replacing the standard layer-wise computation of neural networks with independent, neuron-wise dynamics that interact through an explicit graph topology [1]. The framework was described in a paper submitted to arXiv on 19 Jun 2026 and subsequently withdrawn on 23 Jun 2026 [1]. Conventional sequence models — including RNNs, LSTMs, continuous-time networks, and Transformers — rely on layer-wise dynamics, where all neurons in a layer co-evolve through a shared parameterized operator [2]. TND departs from this design by treating each neuron as an independent dynamical unit. The system is formalized as a directed neuron graph, an interaction operator, and a local dynamics function; collective computation emerges from local interactions rather than from a global operator [3]. The authors, including Borui Cai, argue that rich global behavior in complex dynamical systems often arises from locally evolving units connected through structured connectivity [4]. In TND, nodes instantiate single-neuron dynamics and edges define topological interactions, allowing information to circulate and persist across timesteps [5]. This stands in contrast to the fully connected layer paradigm common in deep learning architectures such as convolutional neural networks, where each neuron in one layer connects to all neurons in the next, a design that can make networks prone to overfitting [7]. TND was instantiated as a discrete-time graph-coupled dynamical system and evaluated on a behavior cloning task in single-player Pong [1]. It was benchmarked against Vanilla RNN, Sparse RNN, LSTM, Closed-form continuous-time neural network (CfC), and Transformer baselines [2]. TND achieved the best catch rate and a mean of 17.47 consecutive catches per round, more than three times that of the strongest baseline [1]. The researchers observed that the neuron-wise design produced smoother hidden state trajectories and better performance on the task [4]. The concept of modeling computation through interacting local units draws on principles seen in network neuroscience, which studies brain structure and function through graph theory and examines how connected brain regions give rise to behavior [8]. The TND authors note that existing neural networks are generally seen as low-quality models of brain function, but their framework explicitly adopts the idea that connectivity allows information to persist and self-organize [5][6].