Abstract:Diffusion Policy is a powerful technique tool for learning end-to-end visuomotor robot control. It is expected that Diffusion Policy possesses scalability, a key attribute for deep neural networks, typically suggesting that increasing model size would lead to enhanced performance. However, our observations indicate that Diffusion Policy in transformer architecture (\DP) struggles to scale effectively; even minor additions of layers can deteriorate training outcomes. To address this issue, we introduce Scalable Diffusion Transformer Policy for visuomotor learning. Our proposed method, namely \textbf{\methodname}, introduces two modules that improve the training dynamic of Diffusion Policy and allow the network to better handle multimodal action distribution. First, we identify that \DP~suffers from large gradient issues, making the optimization of Diffusion Policy unstable. To resolve this issue, we factorize the feature embedding of observation into multiple affine layers, and integrate it into the transformer blocks. Additionally, our utilize non-causal attention which allows the policy network to \enquote{see} future actions during prediction, helping to reduce compounding errors. We demonstrate that our proposed method successfully scales the Diffusion Policy from 10 million to 1 billion parameters. This new model, named \methodname, can effectively scale up the model size with improved performance and generalization. We benchmark \methodname~across 50 different tasks from MetaWorld and find that our largest \methodname~outperforms \DP~with an average improvement of 21.6\%. Across 7 real-world robot tasks, our ScaleDP demonstrates an average improvement of 36.25\% over DP-T on four single-arm tasks and 75\% on three bimanual tasks. We believe our work paves the way for scaling up models for visuomotor learning. The project page is available at <a class="link-external link-http" href="http://scaling-diffusion-policy.github.io" rel="external noopener nofollow">this http URL</a>.

Unifying Diffusion Models with Action Detection Transformers for Multi-task Robotic Manipulation

Diffusion Transformer Policy

3D Diffuser Actor: Policy Diffusion with 3D Scene Representations

Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation

RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks

RoLD: Robot Latent Diffusion for Multi-task Policy Modeling

The Ingredients for Robotic Diffusion Transformers

Discrete Policy: Learning Disentangled Action Space for Multi-Task Robotic Manipulation

MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusion

Scaling Diffusion Policy in Transformer to 1 Billion Parameters for Robotic Manipulation

M2Diffuser: Diffusion-based Trajectory Optimization for Mobile Manipulation in 3D Scenes

Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training

Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals

Hierarchical Diffusion Policy: manipulation trajectory generation via contact guidance

EC-Diffuser: Multi-Object Manipulation via Entity-Centric Behavior Generation

Diffusion Policy: Visuomotor Policy Learning via Action Diffusion

Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation

RobotDiffuse: Motion Planning for Redundant Manipulator based on Diffusion Model

SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution