From Values to Tokens: An LLM-Driven Framework for Context-aware Time Series Forecasting via Symbolic Discretization

A research team has proposed TokenCast, a framework that uses large language models to improve time series forecasting by converting numerical data into a symbolic token format the model can process alongside unstructured text [1]. The framework, detailed in a paper posted to the arXiv preprint server, targets a persistent weakness in forecasting systems: the difficulty of merging historical numerical sequences with contextual features that often arrive as unstructured text [1]. Time series forecasting supports decision-making in energy, healthcare, and finance, but accuracy has been constrained by this modality gap [2]. TokenCast addresses the problem through a three-stage pipeline. First, a discrete tokenizer transforms continuous numerical sequences into temporal tokens, creating a structural alignment with language-based inputs [2]. Both temporal and contextual tokens are then embedded into a shared representation space using a pre-trained large language model, or LLM — a type of machine learning model with many parameters trained on vast amounts of text [11]. The aligned LLM is subsequently fine-tuned in a supervised manner to predict future temporal tokens, which a decoder converts back into the original numerical space [2]. Large language models are built on transformer architectures that use attention mechanisms to model long-range dependencies in data [3]. Applying them to time series forecasting has drawn interest because the same architectures that process language can, with appropriate tokenization, learn patterns in sequential numerical data [3]. The TokenCast authors argue that casting both modalities into a shared symbolic space allows the model to exploit contextual signals — such as weather reports or policy announcements — that traditional forecasting models ignore [2]. The paper was submitted to arXiv on 8 August 2025, with a revised version posted on 17 June 2026 [1]. The first submission weighed 2,675 KB; the second came in at 1,996 KB [1]. The sole listed author is Xiaoyu Tao, and the code has been released on GitHub [2]. arXiv, which hosts the paper, is an open-access repository of electronic preprints that are moderated but not peer-reviewed; it was founded in 1991 and now receives roughly 24,000 submissions per month [9]. The authors report extensive experiments on real-world datasets, though the paper has not yet appeared in a peer-reviewed journal [2]. High-quality labeled training datasets are typically expensive to produce in machine learning, making publicly available benchmarks important for reproducibility [5]. The TokenCast code release follows a pattern common in the machine learning community, where linking papers to code and data has become a standard practice facilitated by tools such as arXiv’s Links to Code & Data feature [8].