Robots that reason as fast as they move have been the missing piece in physical AI. Google DeepMind just took a serious step toward closing that gap with Gemini Robotics ER 2, an embodied reasoning model that plans multi-step tasks at sub-second latency and watches live video to track its own progress. The model launched July 30, 2026 and is available now through the Gemini API, Google AI Studio, and private preview on the Gemini Enterprise Agent Platform.
Google reports 91.3% accuracy on moment-finding tasks at 0.96 seconds mean absolute distance. The model handles reasoning, tool-calling, and task planning, then hands motor execution to any lower-level vision-language-action model you choose. Think of it as an orchestration layer that sits between your natural language commands and your robot's actuators. The same announcement claims 4x execution speed compared to much larger frontier model categories, putting ER 2 in the sub-second range that real-world robotics demands.
What did Google actually ship?
The architecture is the interesting part. Gemini Robotics ER 2 is built on Gemini 3.5 Flash, according to the model card Google published alongside the launch. That base model choice explains the speed: Flash-class models trade parameter count for latency, and in robotics, latency is safety. A model that takes three seconds to decide whether to stop pouring coffee is a model that spills.
The model ingests continuous video feeds, audio, and text through the Gemini Live API, a bidirectional streaming endpoint designed for latency-sensitive tasks. It uses that stream to maintain situational awareness, classify how far along a task is, and decide when to move to the next step. Google frames it as a physical agent that orchestrates steps, self-corrects, and generalizes to novel situations.
Developers declare low-level control interfaces as tools. You can wire up a VLA model, a navigation stack, or a manipulator API as a callable function, and let ER 2 decide when to invoke which one. Google demonstrated this with Boston Dynamics' Spot, using ER 2 to orchestrate navigation and manipulator movement so the robot could fetch objects on natural language commands. The example code is available on GitHub. They also showed multi-robot collaboration with Apptronik's Apollo 2 humanoid and the Franka F3 Duo arm, where different robots communicated through a shared semantic understanding to hand off tasks.
The model can also natively call Google Search and user-defined functions, which means a robot could look up assembly instructions mid-task or query a parts database without a human in the loop. That sounds like a convenience feature, but it has real implications for how you architect a robotics pipeline. If your robot can search the web for a manual when it encounters an unfamiliar object, your task definitions can be far less rigid.
How good are the benchmark numbers really?
Here is where you should pay close attention. Google published two specific accuracy figures, and they tell very different stories.
Moment-finding accuracy hit 91.3% with a 0.96-second mean absolute distance. That means the model can identify the exact video frame where a critical event happens, like when to stop pouring or when a bolt is tight, and it is off by less than a second on average. Google says this rivals much larger model categories at a fraction of the compute cost. For a task where the difference between a full cup and an overflow is a quarter second, that is genuinely useful.
Progress classification landed at 57.4% accuracy. This metric tracks how well the model can bucket a video frame into one of five progress levels, from 0 to 20% up to 80 to 100%. Fifty-seven percent sounds mediocre until you realize random chance on a five-class problem is 20%. But it also means the model misclassifies progress more than 42% of the time, which is a real limitation for any workflow that depends on knowing exactly where a task stands.
The chart below puts both benchmarks side by side.
Ars Technica's coverage of the launch noted that the technology remains far from ready for unsupervised deployment. That framing is fair. A robot that correctly tracks progress slightly more than half the time is a robot that needs a human watching the other half.
Google also claims ER 2 outperforms ER 1.6 and other frontier models on Safety Instruction Following and Human Proximity benchmarks, and that it successfully halts a humanoid robot when a person is nearby and resumes only when the area is clear. These are important capabilities. Google has not published specific accuracy numbers for them, though a safety technical report is available for those who want to dig deeper. The safety claims are directional, and you should treat them accordingly.
Why does this matter for someone building robots?
The architecture shift is the real story for builders. Here is what changes:
-
You still need a VLA model. ER 2 is a reasoning and orchestration layer. Your VLA model handles motor control. The separation means you can swap VLA models without retraining the reasoning layer, which is a meaningful design decision for anyone maintaining a robotics stack.
-
Streaming is the integration pattern. The Gemini Live API bidirectional streaming endpoint is how ER 2 maintains real-time awareness. If your robotics stack is built on request-response patterns, you will need to rearchitect for continuous streaming. This is a different integration model, and it affects your latency budget, your error handling, and your connection management.
-
Multi-robot coordination is now a platform feature. ER 2 enables robots to communicate via shared semantic understanding. You can build workflows where a wheeled rover handles indoor navigation and a humanoid handles manipulation, with ER 2 orchestrating the handoff. This has been a research problem for years, and Google is shipping it as an API feature.
-
Safety compliance might get easier. ER 2's human proximity detection and safety instruction following could reduce the burden on your safety-critical code. Without published accuracy numbers, you should treat it as a helpful signal. If you are operating near humans, you still need hardware safety interlocks.
For a sense of where the broader field is heading, the LeRobot DAgger correction flywheel we covered earlier shows how robot learning is converging on continuous feedback as the core training paradigm. ER 2 applies the same idea at inference time: watch the video, track progress, correct course.
What should you do with it?
If you are already building with Gemini models, the API is open now. Start with a simple orchestration task: wire up one VLA model and one navigation API as tools, and let ER 2 handle task sequencing. The Boston Dynamics Spot demo code on GitHub is a reasonable starting point for understanding the tool-calling pattern.
If you are building a multi-robot system, the collaboration features are worth prototyping. Keep your expectations in check. The demos show two robots handing off tasks in controlled settings. The real test is whether shared semantic understanding holds up when the environment is messy and the robots come from different manufacturers with different kinematics.
Cost is the open question. Google says ER 2 delivers its precision at a fraction of the compute cost of larger model categories, but has not published specific pricing. The fact that it runs on Gemini 3.5 Flash suggests the per-token economics will be closer to a standard API call than a frontier model inference. You should verify before committing to it as your reasoning layer.
The 57.4% progress classification accuracy is the number to watch in your own testing. If your tasks are simple and well-bounded, the model may perform better than the benchmark suggests. If your tasks involve ambiguous intermediate states, like assembling parts that look similar at different stages of completion, expect the model to struggle. Build retry logic and human-in-the-loop fallbacks into your orchestration code from the start.
The architecture is the bet
Google's roadmap signals are clear: more complex tasks, more robot types, tighter integration between reasoning and action. The competition is not standing still either. Physical AI is becoming a platform war, and the winner will be whoever gives builders the fastest path from natural language command to safe physical execution.
The architecture pattern matters more than any single benchmark. A reasoning layer that streams video, calls tools, and orchestrates multiple robots is a genuinely useful abstraction for anyone building a robotics stack. Whether ER 2's specific accuracy is good enough for your use case is a question only your own testing can answer. The pattern of separating reasoning from action, and streaming rather than polling, is where robotics stacks are heading. Google just handed you the reasoning layer. The rest is still on you.
Sources
- Google DeepMind - Introducing Gemini Robotics ER 2
- Google DeepMind Model Card - Gemini Robotics ER 2 Model Card
- Ars Technica - Google reveals Gemini Robotics 2.0, promising improved dexterity and safety
- Google DeepMind - Gemini Robotics ER 2 product page
