Robots that learn by being shown: vision-language-action models in 2026

What a vision-language-action model is

Google DeepMind coined the term in July 2023 with RT-2, which turned robot actions into text tokens and trained them alongside ordinary language, so a model that had learned from the web could also output a movement. Most VLAs since have the same three parts: a vision encoder that turns camera frames into tokens, a language model that reads them together with the instruction, and an action output that turns the result into joint or gripper commands.

The action output is where designs differ. RT-2 and the open OpenVLA (June 2024, 7 billion parameters, built on Llama 2) predict actions as discrete tokens. Physical Intelligence’s pi 0 (October 2024) instead added flow matching, a variant of diffusion, on top of a pre-trained vision-language model, which produces continuous motion up to 50 times per second, the rate that dexterous tasks such as folding laundry need.

A VLA is a policy: it decides what to do next. That separates it from a world model, which predicts what will happen if the robot moves and leaves the choice of movement to a planner. The two are starting to work together, as the pi 0.7 section below shows.

How VLAs learn from demonstrations

Almost all of the training signal comes from people showing the robot what to do. An operator drives the robot through a task by teleoperation, with a VR headset or a leader arm that the robot copies, and every camera frame is stored with the motor commands that followed it. The model learns to imitate that mapping.

Pooling data across robots is what made generalist models possible. The Open X-Embodiment collection, published in October 2023 by 21 institutions, combined data from 22 robot types covering 527 skills, and a model trained on all of it improved several robots by drawing on the others’ experience. The pattern since then is the same: pre-train on many robots and web data, then fine-tune on the target robot. In June 2025 Google DeepMind said its first on-device Gemini Robotics model adapted to new tasks with as few as 50 to 100 demonstrations.

Teleoperation is slow, so labs top it up with simulation. When NVIDIA launched GR00T N1 in March 2025, it said it had generated 780,000 synthetic trajectories in 11 hours, the equivalent of about 6,500 hours of human demonstrations, and that combining synthetic and real data improved the model by 40% over real data alone. The catch is the gap between simulated and real physics, so a result on a physical robot counts for more than a simulation score.

Many current systems split the work into a slow and a fast part. GR00T N1 pairs a slow vision-language model that reasons about the scene and plans with a fast action model that turns the plan into continuous movement. Gemini Robotics 2 makes the same split with a separate reasoning model.

Gemini Robotics 2: three models, one brain

Google DeepMind’s first Gemini Robotics models, built on Gemini 2.0, appeared in March 2025, and version 1.5 in September 2025 added reasoning shown before each action and transfer of motions learned on one robot to another. On 30 July 2026 it introduced three successors that work together. Gemini Robotics 2 is the action model that turns vision and language into motor control, including whole-body control of humanoids. Gemini Robotics ER 2 is the embodied reasoning model that plans multi-step tasks lasting minutes, tracks progress and self-corrects. Gemini Robotics On-Device 2 runs locally on the robot without a network.

The success rates published with it are specific. On an Apptronik Apollo humanoid with five-fingered hands: picking an object from a table 68.4%, from a shelf 76.3%, from the floor 45.7%. On two-finger grippers: general pick and place 74.2%, diverse tool kitting 78.9%, precise insertion 89.6%. On multi-finger dexterity, which DeepMind calls out as still difficult: unscrewing a bulb 92% but screwing one in 36%, tying a bin bag 44%, sweeping with a dustpan 32%.

Two other details matter for anyone planning a deployment. The on-device model adapts to a new two-arm robot in a few hours, typically with under 200 examples, even when the hardware differs in shape, sensors and degrees of freedom. And safety has its own benchmark, ASIMOV-Agentic, which tests whether the reasoning model refuses unsafe tool calls, predicts whether a task is possible at all and asks a human when it is uncertain. DeepMind lists Apptronik, Boston Dynamics and Agile Robots as partners and more than 100 trusted testers.

pi 0.7: the recipe, not the architecture

Physical Intelligence released pi 0.7 on 16 April 2026. The architecture is modest: Google’s open Gemma 3 4B model paired with an 860 million parameter action expert that generates motions. The company’s claim is that the training recipe decides the outcome.

Instead of a bare task description, the model is trained with extra context: subtask instructions in natural language, metadata about the quality and speed of each demonstration, control mode labels, and subgoal images produced at runtime by a lightweight world model. Failed or sloppy demonstrations are tagged rather than thrown away, and the ablations show why: without quality annotations, adding lower-quality data makes the model worse, while with them the model keeps improving as average quality drops.

The headline result is cross-robot transfer. A two-armed UR5e industrial manipulator folded t-shirts with an 80% success rate although no laundry data had been collected on that robot, which the company compares to experienced human teleoperators trying the task on it for the first time. A single pi 0.7 model also matches the reinforcement-learning-tuned specialists from its previous generation on laundry folding, espresso making and box building.

The open stack caught up

NVIDIA’s GR00T 1.7, a 3 billion parameter vision-language-action model for humanoids, landed in Hugging Face’s LeRobot on 7 July 2026 together with Isaac Teleop, a framework for capturing demonstrations from VR headsets or leader arms in a format training pipelines can read. On the LIBERO benchmark suite the jump from the previous version is reported as 87% to 96.5% average, with the spatial suite going from 82% to 95%.

The practical consequence is that the whole loop is now available off the shelf: record demonstrations, fine-tune a published model, run it on the robot. NVIDIA pairs it with Jetson Thor compute on board, and also offers a reference humanoid platform for academic labs.

What the numbers do not say

Read the claims carefully, because the field has no shared benchmark for manipulation generalization. Every comparison above is run by the lab that built the model, usually against its own previous version.

• Independent comparisons are rare and older. A 2024 study that ran OpenVLA, GPT-4o and JAT on 20 datasets from Open X-Embodiment found performance varied widely across tasks and robots, and every model struggled with manipulation that needs multi-step planning.
• Picking off the floor at 45.7% is not a production process. Dexterity numbers in the 30% to 44% range are research results, not deployments.
• Physical Intelligence concedes that with a dataset this large and diverse it is hard to know which tasks are genuinely novel, so claims of compositional generalization need independent replication.
• Access is limited: the action models are with early-access partners, while the reasoning model is the part developers can use in Google AI Studio today.
• Speed is a constraint of its own. A model that must send commands dozens of times a second cannot wait on a cloud round trip, which is why DeepMind ships an on-device model and NVIDIA pairs GR00T with onboard compute.
• None of these results remove the need for safety engineering around a machine that moves. The one benchmark aimed at that, ASIMOV-Agentic, was only published in July 2026.

What this means in practice

The shift for industry is about where value sits. If a published model plus a few hundred demonstrations gets you most of the way to a new embodiment, then the scarce asset is not the model: it is demonstration data from your own process, captured cleanly, labeled with how good each attempt was, and kept. That is exactly what Isaac Teleop and the pi 0.7 metadata approach are for, and it is the first thing we set up in a physical AI project.

In our own humanoid robot training the robot learns on a safety hoist, one movement at a time, with a person approving every new skill before it is used. That sequence, demonstrate, evaluate, approve, is what we would build around any of the three models above. If you have a repetitive station where the work is physically simple and the variation is small, that is the kind of task worth a two-week proof of concept.