Abstract:Large policies pretrained on a combination of Internet-scale vision-language data and diverse robot demonstrations have the potential to change how we teach robots new skills: rather than training new behaviors from scratch, we can fine-tune such vision-language-action (VLA) models to obtain robust, generalizable policies for visuomotor control. Yet, widespread adoption of VLAs for robotics has been challenging as 1) existing VLAs are largely closed and inaccessible to the public, and 2) prior work fails to explore methods for efficiently fine-tuning VLAs for new tasks, a key component for adoption. Addressing these challenges, we introduce OpenVLA, a 7B-parameter open-source VLA trained on a diverse collection of 970k real-world robot demonstrations. OpenVLA builds on a Llama 2 language model combined with a visual encoder that fuses pretrained features from DINOv2 and SigLIP. As a product of the added data diversity and new model components, OpenVLA demonstrates strong results for generalist manipulation, outperforming closed models such as RT-2-X (55B) by 16.5% in absolute task success rate across 29 tasks and multiple robot embodiments, with 7x fewer parameters. We further show that we can effectively fine-tune OpenVLA for new settings, with especially strong generalization results in multi-task environments involving multiple objects and strong language grounding abilities, and outperform expressive from-scratch imitation learning methods such as Diffusion Policy by 20.4%. We also explore compute efficiency; as a separate contribution, we show that OpenVLA can be fine-tuned on consumer GPUs via modern low-rank adaptation methods and served efficiently via quantization without a hit to downstream success rate. Finally, we release model checkpoints, fine-tuning notebooks, and our PyTorch codebase with built-in support for training VLAs at scale on Open X-Embodiment datasets.

What problem does this paper attempt to address?

The paper aims to address two key issues in the field of robotic manipulation: 1. **Generalization Ability of Universal Robotic Policies**: Current robotic policies often perform poorly when faced with new situations outside of the training data, such as new objects, scene disturbances, or unseen task instructions. Although existing vision-language models (such as CLIP, SigLIP, and Llama 2) can be pre-trained on large-scale internet data, thereby possessing good generalization capabilities, applying these models to the field of robotic manipulation remains a challenge. 2. **Open Source and Efficient Fine-Tuning**: Most existing vision-language action (VLA) models are closed and do not support effective fine-tuning. This means that researchers find it difficult to modify or adapt them to new robotic environments and tasks. Additionally, existing work does not provide best practices for deploying and adapting VLA on common hardware (e.g., consumer-grade GPUs). To address the above issues, the paper proposes OpenVLA—a 700 million parameter open-source vision-language action model. Based on a large-scale and diverse dataset of robotic demonstrations (containing 970,000 robotic demonstration instances), this model demonstrates stronger generalization ability and higher task success rates compared to existing closed-source models (such as RT-2-X). Furthermore, the paper explores efficient fine-tuning strategies and demonstrates how to use low-rank adaptation (LoRA) and model quantization techniques to achieve performance uncompromised model fine-tuning on consumer-grade GPUs. Finally, the authors have made all models, deployment and fine-tuning notebooks, and training code repositories publicly available to promote future research in this area.

OpenVLA: An Open-Source Vision-Language-Action Model

TinyVLA: Towards Fast, Data-Efficient Vision-Language-Action Models for Robotic Manipulation

Benchmarking Vision, Language, & Action Models on Robotic Learning Tasks

CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation

LLaRA: Supercharging Robot Learning Data for Vision-Language Policy

Run-time Observation Interventions Make Vision-Language-Action Models More Visually Robust

ReVLA: Reverting Visual Domain Limitation of Robotic Foundation Models

LADEV: A Language-Driven Testing and Evaluation Platform for Vision-Language-Action Models in Robotic Manipulation

Robotic Control via Embodied Chain-of-Thought Reasoning

Vision-Language Foundation Models as Effective Robot Imitators

LLARVA: Vision-Action Instruction Tuning Enhances Robot Learning

Diffusion-VLA: Scaling Robot Foundation Models via Unified Diffusion and Autoregression

QUAR-VLA: Vision-Language-Action Model for Quadruped Robots

From Goal-Conditioned to Language-Conditioned Agents via Vision-Language Models

KALIE: Fine-Tuning Vision-Language Models for Open-World Manipulation without Robot Data

A Dual Process VLA: Efficient Robotic Manipulation Leveraging VLM

Towards Testing and Evaluating Vision-Language-Action Models for Robotic Manipulation: An Empirical Study

3D-VLA: A 3D Vision-Language-Action Generative World Model

VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Models

LLaVA-o1: Let Vision Language Models Reason Step-by-Step

A3VLM: Actionable Articulation-Aware Vision Language Model