Human-like autonomy emerges from self-play and a pinch of human data

A new method trains autonomous driving policies using self-play reinforcement learning and just 30 minutes of human demonstration data, a fraction of what imitation learning systems require, according to research published on arXiv [1]. The approach, detailed in a paper submitted June 11, treats human demonstrations as a regularization objective layered atop a minimal safe goal-reaching reward, rather than as the primary training signal [1]. Self-play reinforcement learning has gained traction because it substitutes cheap, large-scale simulations for expensive human driving data, but policies trained this way often develop driving conventions that are effective yet incompatible with human behavior [2]. Previous mitigation attempts relied on extensive reward engineering and domain randomization, which the authors describe as brittle and labor-intensive [2]. "Like the spice in a good stew, we find that a little human data goes a long way," the researchers write [2]. The method uses 30 minutes of human demonstrations — 2,500 times fewer than comparable imitation learning approaches — and the resulting policies coordinate with held-out human trajectories while completing training in 15 hours on a single consumer-grade GPU [2]. The paper appears on arXiv, the open-access repository operated by Cornell University that hosts preprints in physics, mathematics, computer science, and related fields. arXiv has increasingly integrated interactive tools to make research more accessible. Since November 2022, arXiv has collaborated with Hugging Face to embed machine learning demos directly alongside papers through a dedicated Demos tab, allowing readers to try models in a browser without writing code [6][7]. Hugging Face Spaces, launched in October 2021, hosts over 12,000 community-built demos using open-source libraries such as Gradio and Streamlit [6]. Authors can link a Space to their paper by including the paper's URL in the Space's README file or by associating a model on the Hugging Face Hub with the paper [8]. The research was submitted under the Machine Learning category and is associated with arXivLabs, a framework that lets community collaborators develop and share new features on the arXiv platform [1]. The authors have released videos and full source code alongside the paper [2].