FUTO Swipe: Layout-Agnostic Neural Swipe Decoding

A new neural swipe decoder can operate on any contiguous mobile keyboard layout without retraining, according to a technical report published on arXiv. The model, called FUTO Swipe, combines the flexibility of algorithmic decoders with the accuracy of neural approaches. [1] Neural swipe decoders are typically tied to the keyboard they were trained on, requiring a new corpus and training run for each layout. [1] The FUTO Swipe encoder instead predicts whether a user is indicating a character and where on the keyboard that character lies at each point along the swipe. The keyboard layout is supplied at inference time and used to map the spatial and temporal prediction to a logit at each key, rather than being learned during training. [1] Training neural models requires substantial data, but public swipe data is limited, particularly for non-QWERTY layouts. [1] The researchers released swipe.futo.org, a corpus containing over 1M donated swipes from more than 12k donor sessions, licensed under the MIT license. [1] The dataset is hosted on Hugging Face and contains multiple collection runs. The initial run, swipe-1, includes around 1 million swipes drawn from Wikipedia sentences sourced from Mozilla Common Voice. Four smaller subsequent runs added between roughly 28,000 and 59,000 swipes each. [5] To generalize beyond the English QWERTY layout, the team applied geometric augmentations to both the swipe trajectory and the keyboard layout at every training step. This forces the model to make predictions based on characteristics of the swipe gesture rather than the training layout. [1] The encoder consumes the keyboard layout at inference as a tensor of (x,y) coordinates for each key, and the spatial output head reads those coordinates through a basis supplied at runtime, rather than learning a separate parameter for each key. [3] The model generalizes to layouts absent from training, in some cases more accurately than the layout it was trained on. [1] The encoder, codenamed "honorable_sturgeon," is a 1D temporal convolutional network with 635,000 parameters. It reads a raw (x, y) touch trajectory resampled to 64 points and emits a 64-coefficient spectral pattern and a scalar intention gate for each timestep. Per-key character scores are read off by evaluating a fixed cosine basis at the layout key centers. [4] Two optional companion models are also available. A decoder, "magic_macaw," refines the character distribution on specific layouts using frozen encoder features; currently, training data exists only for English QWERTY. A context language model, "hungry_jellyfish," provides next-word and beam-rerank capabilities for English. [4] All trained models are publicly available. [1]