Hugging Face just shipped LeRobot v0.6.0, and the headline is not another model zoo. It is a closed loop: deploy a policy, watch it fail, correct it with your hands, and fine-tune on the corrections, all from a single CLI. The release adds five new vision-language-action models, three world model policies, two reward models, six simulation benchmarks, depth sensing, automatic VLM-powered dataset annotation, and cloud training. But the part that changes how a builder works is the lerobot-rollout CLI with DAgger-style human-in-the-loop corrections, which turns deployment into data collection without leaving the terminal.
For anyone building with physical robots, the gap between a policy that works in simulation and one that works on hardware has always been the expensive part. You train, you deploy, it fails in ways the simulator never predicted, and then you spend weeks collecting fresh demonstrations. LeRobot v0.6.0 compresses that cycle into a command. If you want the broader read on how this release fits the robot learning flywheel, check our earlier analysis of the v0.6 loop. Here we dig into what the pieces actually do and what you should do with them.
What actually shipped in LeRobot v0.6.0?
The release is large. The Hugging Face team grouped it into six areas, each with concrete deliverables. Here is what is new, with the numbers that matter.
World model policies. Three policies that learn to imagine future frames during training, each with a different strategy for keeping inference cheap. VLA-JEPA, built on Qwen3-VL-2B, predicts future frames in latent space during training and then discards the world model at inference, so you get world-model supervision at zero extra inference cost. LingBot-VA predicts future video and actions together autoregressively, chunk by chunk, and runs on a single 24 to 32 GB GPU. FastWAM pairs a roughly 5B video-generation expert with a compact action expert, dreams rollouts during training, and skips the dreaming at inference to denoise action chunks directly.
New VLAs. Five new vision-language-action models join the zoo. GR00T N1.7 upgrades NVIDIA's cross-embodiment model to Cosmos-Reason2-2B on Qwen3-VL with a flow-matching action head, and the integration is parity-tested against NVIDIA's original Isaac-GR00T implementation. MolmoAct2 from the Allen Institute for AI runs inference in about 12 GB at bf16 and LoRA fine-tunes on a single 24 GB GPU. EO-1 uses a Qwen2.5-VL-3B backbone. Multitask DiT is a 450M-parameter diffusion transformer from TRI. EVO1 packs into 0.77B parameters on an InternVL3-1B backbone, small enough for real-time inference on modest GPUs.
Reward models. A unified API (lerobot.rewards) with four reward models. The standout is Robometer, a 4B-parameter pretrained model built on Qwen3-VL-4B and trained on more than one million robot trajectories via trajectory comparisons. Point it at any LeRobot dataset and it scores task progress and success from raw video plus a language instruction, with no task-specific training. TOPReward goes further: it wraps an off-the-shelf VLM and reads the log-probability of the token "True" given the trajectory video and the task instruction. No reward weights at all.
Benchmarks. Six new simulation benchmarks runnable through a single lerobot-eval CLI: LIBERO-plus (roughly 10,000 perturbed variants across seven axes), RoboTwin 2.0 (50 bimanual tasks with 100k trajectories), RoboCasa365 (365 kitchen tasks in 2,500 procedurally generated kitchens), RoboCerebra (long-horizon chains of 3 to 6 sub-goals), RoboMME (16 memory tasks), and VLABench (knowledge and reasoning in manipulation). Combined with the existing LIBERO, Meta-World, and NVIDIA IsaacLab-Arena, that is nine benchmark families under one roof.
Dataset improvements. Training on video datasets is now up to 2x faster, and loading a subset of a large dataset dropped from 275 seconds to 0.06 seconds in the team's benchmark. Depth support works end to end with Intel RealSense, captured in millimeters and compressed as 12-bit depth video. A new lerobot-annotate CLI uses a VLM to automatically fill in rich language annotations: timestamped subtasks, plans, corrections, and per-camera VQA pairs.
Training and deployment. FSDP support lets you shard models across GPUs and checkpoint to a single safetensors file. Cloud training on HF Jobs runs the same lerobot-train command with one flag, on anything from a T4 to 8x H200.
How does the DAgger correction loop actually work?
This is the part a builder should care about most. The new lerobot-rollout CLI replaces what used to be a hack on top of lerobot-record. It supports five strategies: base (just run the policy), sentry (continuous recording with auto-upload), highlight (ring buffer that saves the last N seconds on keypress), episodic (classic episode/reset), and dagger (deployment as data collection).
Here is the DAgger workflow in practice. You start the policy on your robot. You watch it run. The moment it goes wrong, you hit a key or a USB foot pedal, take over with your leader arm to record the correction, and hand control back. Actuated leaders are driven to the follower's pose before you take over, so the handover is smooth. Every correction frame gets an intervention flag, and the resulting dataset is ready for fine-tuning.
The command looks like this:
lerobot-rollout \
--strategy.type=dagger \
--policy.path=${HF_USER}/my_policy \
--robot.type=so100_follower \
--teleop.type=so101_leader \
--teleop.port=/dev/ttyUSB0 \
--dataset.repo_id=${HF_USER}/dagger_corrections \
--dataset.single_task="Grasp the block" \
--push_to_hub=trueThen you fine-tune on the corrections, redeploy, and repeat. The flywheel is a CLI flag.
The reason this matters is that the bottleneck in real-robot learning has never been model architecture. It has been data. Specifically, it has been the cost of collecting targeted corrections when a policy fails in ways the training distribution did not cover. Simulation benchmarks like LIBERO-plus, with its 10,000 perturbed variants, tell you where your policy breaks. DAgger tells you what to do about it: record the fix, tag it, and feed it back.
Do world models actually help robot policies?
The Hugging Face team frames this as the open question, and v0.6.0 brings three different bets to answer it. Each takes a different approach to the same problem: imagining the future is computationally expensive, and you do not want that expense at inference time.
VLA-JEPA trains a JEPA world model to predict upcoming frames in latent space from the policy's own actions, then drops the world model entirely at inference. You get the supervision signal during training and pay nothing at deploy time. This is the cleanest answer to the cost problem.
LingBot-VA predicts future video and actions together, chunk by chunk, and feeds real observations back in to stay grounded. You can save what the robot imagined and compare it with what actually happened, which is useful for debugging but means you are paying for video generation at inference.
FastWAM pairs a roughly 5B video-generation expert with a compact action expert. During training the model learns to dream its own rollouts. At inference it skips the dreaming and directly denoises action chunks. The paper's title asks whether world action models need test-time future imagination at all, and FastWAM's answer is no.
The practical read: if you are choosing between these three, VLA-JEPA and FastWAM both give you world-model training benefits without inference overhead. LingBot-VA is the one to pick if you want to inspect what the model imagines for debugging, and you have a 24 to 32 GB GPU to spare.
What do reward models buy me that I did not have before?
Success detection has been a gap in the robot learning loop. You train a policy, you deploy it, and then you need to know whether it succeeded. Without an automated success detector, you either watch every episode yourself or you rely on simulator ground truth that does not transfer to the real world.
Robometer changes this. It is a 4B model trained on more than one million robot trajectories that scores task progress and success from raw video plus a language instruction. No task-specific training. You point it at your dataset and get per-frame progress curves. That is useful for three things:
- Reward-aware behavior cloning (RA-BC): use the progress curves as a reward signal to weight demonstrations during training.
- Dataset quality inspection: find episodes where the policy made no progress, or where the task was never completed, and flag them for review or removal.
- Progress-overlay videos: generate visualizations that show per-frame progress, which is useful for demos and for debugging where a policy starts to degrade.
TOPReward is the zero-shot version. It wraps any capable VLM and reads the log-probability of the token "True" given the trajectory video and the task instruction. No reward weights, no training. The tradeoff is that you are relying on a general-purpose VLM's judgment of whether a robot task succeeded, which is a weaker signal than a model trained specifically on robot trajectories. But it costs you nothing to try.
For a team building a robot product, the value is concrete. You can automate success detection across thousands of real-world deployment episodes without a human in the loop for every one. That scales the evaluation side of the flywheel to match the data collection side.
What should I do with this release?
If you are already using LeRobot, upgrade. The installation is leaner, data loading is 2x faster, and the lerobot-rollout CLI replaces the hack you were running on top of lerobot-record. Here is what to do next, depending on where you are.
If you are training policies on a single GPU: look at EVO1. At 0.77B parameters on an InternVL3-1B backbone, it is small enough to fine-tune and run in real time on modest hardware. It ships with two-stage fine-tuning and Real-Time Chunking support. Multitask DiT at 450M parameters is another option if you want a diffusion-based policy you can train yourself.
If you have a 24 GB GPU: MolmoAct2 runs inference in about 12 GB at bf16 and LoRA fine-tunes on a single 24 GB GPU. It comes with ready-made checkpoints calibrated for the SO-100 and SO-101 arms, so you can deploy zero-shot.
If you have multiple GPUs: FSDP support means you can shard parameters, gradients, and optimizer state across GPUs and checkpoint to a single safetensors file. You can resume on a different number of GPUs than you started on. Cloud training on HF Jobs lets you run the same lerobot-train command on anything from a T4 to 8x H200 with one flag.
If you are deploying on real hardware: set up the DAgger loop. Start with the base strategy to run your policy, switch to dagger when you want to collect corrections, and fine-tune on the flagged intervention frames. This is the single most valuable workflow in the release.
If you are evaluating policies: run the six new benchmarks through lerobot-eval. Start with LIBERO-plus if you want to know where your policy breaks under distribution shift. Start with RoboCerebra if you care about long-horizon tasks. Each benchmark ships with a Docker image and a SmolVLA baseline checkpoint smoke-tested in CI.
If you are annotating datasets: try the lerobot-annotate CLI. It uses a VLM to watch your episodes and fill in timestamped subtasks, plans, corrections, and per-camera VQA pairs. A YAML recipe layer renders these into chat-style training messages at sample time, which is the format long-horizon policies will train on.
What is still missing?
The release is comprehensive, but gaps remain. The DAgger loop assumes you have a leader arm for teleoperation. If you are working with a robot that does not have a compatible leader, you are back to manual demonstration collection. The depth support covers Intel RealSense and a list of arms including SO-100/101, Koch, OpenArm, reBot, and Unitree G1, but it is not universal across all hardware.
The reward models are new. Robometer was trained on more than one million trajectories, but it is still a single model with a specific training distribution. How well it generalizes to tasks and embodiments outside that distribution is an open question that the release blog does not answer. TOPReward depends entirely on the VLM you point it at, and VLM judgments of physical task success are a noisy signal.
The six new benchmarks are all simulation. They tell you where your policy breaks in simulation, which is useful, but the transfer gap to real hardware remains. The DAgger loop is the bridge, but it is a human-in-the-loop bridge, not an automated one.
World models are presented as an open question, and the three policies shipped represent three different bets. The release does not claim a winner. It gives you the tools to run the experiment yourself.
The bet to make now
The model zoo will keep growing. The benchmarks will keep multiplying. The part of this release that has a clear, defensible return today is the DAgger correction loop paired with automated reward models. Together they compress the most expensive part of real-robot learning, the cycle between deployment failure and improved training data, into a workflow you can run from a terminal. If you are building with robots, that is where your time goes.
Sources
- Hugging Face Blog LeRobot v0.6.0: Imagine, Evaluate, Improve
- Hugging Face Docs GR00T Policy
- Hugging Face Docs EO-1 Policy
- Hugging Face Docs Multitask DiT Policy
- GitHub huggingface/lerobot
