Test-time Adversarial Takeover: A Real-time Hijacking Interface against Robotic Diffusion Policies

Security researchers have demonstrated a method to hijack robotic policies in real time, turning a frozen robot into a remotely piloted instrument. The technique, called Test-time Adversarial Takeover (TAKO), achieved a 100% takeover success rate across every evaluated setting, according to a preprint posted to arXiv on June 9 [1][2]. The TAKO attack targets the visual conditioning pathway of diffusion-based visuomotor policies, which have become foundational in embodied AI [2]. By learning a small vocabulary of reusable universal patches through differentiable diffusion inference, an attacker can switch among these patches in the camera stream to compose attacker-chosen trajectories [2]. The perturbation induces a bias that persists through iterative generative inference, allowing the attacker to steer the robot [2]. Researchers tested the method across four tasks: 2D manipulation, simulated aerial delivery, simulated ground navigation, and physical-world ground navigation [2]. The attack proved effective against two visual encoders — ResNet-18 and EfficientNet-B0 with a Transformer — and across three generative inference families: DDPM, DDIM, and flow matching [2]. In every configuration, human operators achieved 100% takeover success on attacker-defined objectives [2]. The work highlights a vulnerability distinct from earlier adversarial attacks on robotic systems. Most prior efforts focused on disruption — perturbing the observation stream to reduce task success or cause erratic behavior [2]. TAKO represents a stronger threat model by providing a real-time steering interface over a frozen policy [2]. The researchers also found that a natural targeted baseline, target-policy matching, failed because the victim policy could not reliably supervise itself on out-of-distribution target shifts [2]. The preprint was submitted to arXiv, an open-access repository of electronic preprints that is moderated but not peer-reviewed [6]. As of November 2024, arXiv received approximately 24,000 article submissions per month [6]. The repository, which began in 1991, surpassed two million articles by the end of 2021 [6]. The TAKO project page is available at https://tako-attack.github.io [2].