FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction

A new framework called FlowPipe reformulates the construction of automated data-preparation pipelines as a conditional probabilistic flow problem, outperforming existing methods on 74 real-world datasets, according to research posted to arXiv on June 23 [1]. The work, titled “FlowPipe: LLM-Enhanced Conditional Generative Flow Networks for Data Preparation Pipeline Construction,” targets a persistent bottleneck in machine learning: transforming raw tabular data into learning-ready form through sequences of cleaning and feature-transformation operators [1]. Because operator sequences are combinatorial and end-to-end evaluation is expensive, existing state-of-the-art Multi-DQN methods still suffer from decoupled value estimators that weaken long-horizon credit assignment, weak injection of dataset context into the policy, and inefficient exploration in a sparse search space with many invalid states [1][2]. FlowPipe addresses these limitations by using Conditional Generative Flow Networks with a Trajectory Balance objective, which connects terminal validation rewards directly to early pipeline decisions [1][2]. The framework further introduces Deep Semantic Modulation via Feature-wise Linear Modulation, allowing logical priors derived from large language models to condition the policy’s internal activations according to dataset semantics [1][2]. Large language models are machine-learning models with many parameters, trained with self-supervised learning on vast amounts of text [8]. FlowPipe also incorporates failure awareness into the flow objective to avoid invalid states and concentrate search on high-potential regions [1][2]. Across two benchmark suites covering 74 real-world datasets, FlowPipe improved accuracy by 11.96% on average over state-of-the-art baselines and achieved 12.5x faster training convergence [1][2]. Source code has been made available on GitHub [1][2]. The paper appeared on arXiv, an open-access repository of electronic preprints that, as of November 2024, receives about 24,000 submissions per month and hosts more than two million articles [6]. The abstract page for the paper includes arXivLabs tabs, a framework launched in 2020 that allows community collaborators to develop and share experimental tools such as bibliographic explorers and code finders directly on the article record page [4][5].