ScaleToT: Generalizing Structured LLM Reasoning for Billion-Scale Low-Activity User Modeling

A new method called ScaleToT aims to bring structured large-language-model reasoning to the billion-scale population of low-activity users whose sparse profiles have historically resisted accurate modeling, according to research posted to arXiv [1]. Accurate user modeling in advertising and recommendation systems typically relies on rich interaction histories, but those histories are absent for billions of low-activity users [1]. Large language models, which are machine learning models with many parameters trained on vast amounts of text, can infer latent user states from static profiles, yet that reasoning becomes unreliable when profiles are sparse and running an LLM across a billion-user base is prohibitively expensive [1][11]. The ScaleToT framework addresses both problems by learning structured reasoning from a small subset of users processed by an LLM and then extending it to the broader low-activity population [1]. To improve reliability, ScaleToT constructs typed user-state chains and refines them through a bounded entropy-guided Tree-of-Thought procedure [1]. The resulting teacher-curated chains are used to train a student model on static profiles via supervised fine-tuning and a technique the authors call Outcome-Driven Segment-Aware Implicit Reward Policy Optimization, or OSIPO [1]. The student’s reasoning representations are then transferred to a lightweight profile encoder, which supplies shared reasoning signals for the remaining users without requiring further LLM inference [1]. The researchers evaluated ScaleToT on lifetime-value prediction in a billion-scale advertising deployment [1]. A randomized online A/B test increased a metric labeled LT30 by 6.738%, while offline reasoning covered only 7.32% of the potential population, a design that the paper states greatly reduces compute cost compared with full-population reasoning [1]. The work appears on arXiv, the open-access e-print repository that has hosted more than two million articles since its launch in 1991 and currently receives roughly 24,000 submissions per month [9].