Cross-Domain Energy-Guided Diffusion Generation for Off-Dynamics Reinforcement Learning

A new framework called CEDGE aims to improve how reinforcement learning agents adapt to environments that differ from their training data, using a trajectory-level diffusion generation approach guided by energy functions [1]. The framework, detailed in a paper submitted to arXiv on 24 May 2026, addresses a problem known as off-dynamics offline reinforcement learning, where a policy must be learned for a target domain using a large dataset from a source domain and only a limited dataset from the target, under mismatched transition dynamics [1][2]. Existing methods such as reward augmentation and data filtering are limited because they cannot synthesize new target behavior beyond the collected source trajectories [2]. Model-based approaches that learn target-aware dynamics generate experience only at the transition level, which the authors state leads to accumulated errors over long horizons [2]. CEDGE, which stands for Cross-domain Energy-guided Diffusion GEneration, trains a trajectory diffusion model on source-domain trajectories and then adapts the generated samples to the target domain through energy guidance [1][2]. This guidance is derived by minimizing the distribution mismatch between source and desired target-domain trajectories and is decomposed into three components: return, domain, and behavior energy [2]. The resulting energy-guided trajectories can be used for direct planning or as synthetic data for policy learning [2]. Because target adaptation is achieved via energy guidance rather than retraining the diffusion model, the framework can be efficiently adapted to new target dynamics compared to previous methods [2]. Reinforcement learning is a subfield of machine learning, which itself is a branch of artificial intelligence focused on enabling systems to learn from data and take actions to maximize defined goals [3][4]. Deep learning, a related area, uses multilayered neural networks to perform tasks such as classification and representation learning, with architectures that have been applied across computer vision, speech recognition, and board game programs [5]. The CEDGE framework's use of diffusion models for trajectory generation represents an application of generative modeling techniques that have seen rapid advancement since the 2020s [3]. Experiments on the ODRL benchmark demonstrated that trajectory-level energy-guided generation improves diffusion planning under dynamics shifts and produces synthetic data that improves downstream target policy learning [2]. The paper was submitted under the cs.LG category on arXiv [1].