Beyond Tokenization: Direct Timestep Embedding and Contrastive Alignment for Time-Series Question Answering

A new framework called CADE aims to fix a fundamental flaw in how large language models handle time-series data by embedding each timestep directly into the model’s representation space, bypassing the tokenization step that fragments numerical values [1][2]. Large language models, or LLMs, are machine learning systems trained on vast text corpora for tasks such as language generation [8]. Their application to time-series question answering, known as TSQA, has been limited by a tokenization bottleneck. Standard Byte Pair Encoding splits continuous numerical values into unstable subword tokens whose embeddings lack meaningful metric structure, causing the model to lose information about magnitude, scale, and trend [1][2]. Prior work attempted to mitigate this with patch-based encoders that divide the series into fixed windows, but that approach locks in a single granularity, breaks patterns, and rarely transfers across datasets with different lengths or sampling rates [2]. The proposed framework, Contrastive Alignment with Direct Embedding, or CADE, was detailed in a paper submitted to the arXiv preprint repository on 17 June 2026 [1]. arXiv, founded in 1991, is an open-access repository that hosts electronic preprints across mathematics, physics, computer science, and other fields; it surpassed two million articles by the end of 2021 and now receives roughly 24,000 submissions per month [6]. The CADE paper appears under the Computation and Language category [1]. CADE’s architecture rests on two components. First, a point-wise linear encoder and MLP projector map each timestep directly into the LLM embedding space, preserving exact index-level access and eliminating the need for patching or padding [1][2]. Second, a one-directional supervised contrastive loss aligns the resulting time-series embeddings with frozen class-name text anchors, bridging the semantic gap between numerical and language representations [2]. On the public Time-MQA benchmark, the framework improved performance across six TSQA tasks and outperformed both open-source and proprietary LLM baselines [1][2]. The paper’s abstract page on arXiv also features community-built tools under the arXivLabs initiative, a framework launched in 2020 that lets third-party developers integrate experimental features such as citation explorers and code finders directly on the site [4][5].