Every two years, the gap between what an AI factory can do on one rack and what it could do on the previous generation widens by an order of magnitude. NVIDIA's Vera Rubin NVL72 just made that gap visible in standardized numbers. In its MLPerf Inference v6.1 debut, the rack-scale system delivered up to 3.7x higher throughput than the GB300 NVL72 on the Qwen3-VL benchmark and up to 2.5x on DeepSeek-R1. On agentic workloads measured by SemiAnalysis AgentX, the preview gap stretches to 30x in throughput per megawatt. These are preview results submitted by NVIDIA and pending MLCommons verification, but they tell you where inference economics are heading: each generation of rack-scale silicon is doing more work per watt than the last by a factor that makes power, not GPU count, the binding constraint for anyone building agent infrastructure.
Vera Rubin NVL72 makes its first MLPerf appearance with a throughput lead that reorders the cost-per-token math for inference.
What did Vera Rubin NVL72 actually prove in MLPerf v6.1?
NVIDIA submitted Vera Rubin NVL72 preview results on two of the heaviest benchmarks in the MLPerf Inference v6.1 suite: DeepSeek-R1 and Qwen3-VL. The results are early and pending MLCommons review, but the numbers are concrete.
On Qwen3-VL, Vera Rubin NVL72 delivered up to 3.7x higher throughput than GB300 NVL72 across offline, server, and interactive scenarios, running vLLM with NVIDIA's Dynamo open source inference framework. On DeepSeek-R1, using the TensorRT-LLM library, throughput reached up to 2.5x higher than GB300 NVL72.
The chart above shows the three benchmark multipliers side by side. The Qwen3-VL and DeepSeek-R1 numbers come from MLPerf v6.1. The AgentX figure comes from a separate preview test using SemiAnalysis AgentX, which replays recorded production agentic-coding sessions with real context growth, tool calls, and sub-agent spawning.
The gap between 3.7x on a vision-language benchmark and 30x on an agentic workload is the story. Traditional inference benchmarks measure throughput on a fixed input-output pattern. Agentic workloads are different: context grows across turns, tool calls interrupt generation, sub-agents spawn, and the KV cache swells. Vera Rubin's architecture, with its NVL72 scale-up domain powered by sixth-generation NVLink delivering 10x higher packet rates and 3x lower latency than off-the-shelf Ethernet, is built for that pattern. The NVL72 rack operates as a single accelerator with 72 Rubin GPUs and 36 Vera CPUs, and the interconnect lets disaggregated serving, large-scale expert parallelism, and distributed KV-caching actually function at rack scale.
NVIDIA also submitted results on WAN 2.2, a text-to-video benchmark, where the GB300 NVL72 reached 0.65 720p videos per second at 5.7 seconds per video, which is 9x higher throughput and 7.5x lower latency than a single node. Vera Rubin did not submit WAN 2.2 in this round, but the GB300 result shows the rack-scale architecture already handles video generation efficiently.
How much of the gain is silicon versus software?
A meaningful chunk. NVIDIA's software stack delivered up to 1.6x higher performance on GB300 NVL72 between MLPerf v6.0 and v6.1, specifically on Qwen3-VL, through lower KV cache precision, additional kernel fusion, better kernels, and disaggregated serving with vLLM and Dynamo. The company also reported post-submission gains on GPT-OSS-120B and DLRMv3 that are not yet verified by MLCommons.
This matters because it means the hardware you buy today keeps getting faster. The GB300 NVL72 that shipped months ago is now 1.6x quicker on certain workloads than when it was first benchmarked. If you are depreciating inference hardware over three to four years, continuous software optimization is free performance that drops straight to your cost-per-token line.
The scaling efficiency result is equally important. NVIDIA's DeepSeek-R1 submission scaled from a single GB300 NVL72 rack with 72 GPUs to four racks with 288 GPUs, achieving 99% scaling efficiency in the offline scenario. Throughput grew nearly in proportion to the hardware added. That is rare. Most distributed systems hit diminishing returns well before 4x the nodes. The NVLink scale-up fabric within each rack and the high-bandwidth networking between racks are what make near-linear scaling possible at this size.
For builders running their own inference infrastructure, the techniques driving these gains are architectural patterns you can reason about and, in some cases, replicate with open source tools. Disaggregated serving separates prefill from decode so each phase scales independently. KV-aware routing directs requests to GPUs that already hold the relevant cached context, which is the same principle behind the agent-aware KV cache work that cut serving latency by 45 percent. Fused CUDA kernels like MegaMoE combine computation and inter-GPU communication into a single pass. These are design choices that affect your serving stack regardless of whose silicon you run on.
Does 30x on agentic workloads change the economics?
Yes, and the magnitude is large enough to reshape planning assumptions.
The 30x figure is throughput per megawatt, not raw throughput. It comes from NVIDIA's preview testing on the SemiAnalysis AgentX workload using the DeepSeek V4 Pro model, measured at 160 tokens per second per user. For context, the GB300 NVL72 itself already delivers up to 15x higher throughput per megawatt than the H200 NVL8 on the same AgentX workload, and up to 80x higher on Kimi K3 2.8T.
Stack those numbers and the cumulative improvement from H200 to Vera Rubin on AgentX is approximately 450x in throughput per megawatt. That is a calculation, not a direct claim from NVIDIA: 15x multiplied by 30x. But it frames the trajectory. A data center that could serve a fixed number of agent sessions on an H200 rack can serve roughly 450 times as many on a Vera Rubin rack, within the same power envelope.
The cost numbers track the throughput numbers. NVIDIA reports that Vera Rubin NVL72 delivers up to 35x lower cost per million tokens than GB300 NVL72 on agentic workloads, and roughly one-tenth the cost per million tokens compared to the earlier GB200 NVL72. The GB300 NVL72 already offered up to 10x lower cost per million tokens than H200 NVL8.
For a team building agent products, this has a direct implication. Agentic systems consume up to 15x more tokens than traditional AI applications, as NVIDIA notes, because agents reason, plan, call tools, and iterate across multiple turns. If your agent product burns through tokens at 15x the rate of a chatbot, the cost per token is the metric that determines whether you can ship. The 89% enterprise agent pilot failure rate we covered earlier is driven in part by inference costs that make production deployment uneconomical. Hardware that cuts cost per token by 35x does not solve every problem in that pipeline, but it moves the break-even point for a class of agent workloads that were too expensive to run at scale six months ago.
Should you plan your inference roadmap around Vera Rubin?
Treat the numbers as directional, not contractual. Three caveats matter.
First, the Vera Rubin MLPerf results are preview submissions, pending MLCommons verification. The AgentX results are pending SemiAnalysis review. NVIDIA measured these itself. The numbers are plausible given the architectural changes, but you should wait for verified results before modeling them into a capacity plan.
Second, the full Vera Rubin platform is a seven-chip architecture: Rubin GPUs, Vera CPUs, Groq 3 LPX, NVLink 6 Switch, BlueField-4 DPU, Spectrum-6 SPX, and ConnectX-9 SuperNIC. The AgentX preview results do not yet reflect Vera CPU performance for tool calling, which means the 30x figure may understate or overstate the real-world gain depending on how the CPU handles tool execution in production agent sessions. You are buying a system whose performance depends on all seven pieces working together under your workload mix.
Third, power is now the constraint that determines how much inference you can run. If your data center has a fixed power budget, throughput per megawatt is the number that matters. If you can add power, raw throughput per rack matters more. The grid stability issues that AI data centers have already triggered in places like PJM mean that adding power is getting harder, not easier. Vera Rubin's efficiency gains are timed for a market where power is the binding constraint.
What to watch:
- MLPerf Endpoints benchmark: NVIDIA mentioned an upcoming MLPerf benchmark specifically for agentic inference workloads. When that lands, it will give you a standardized, verified number for agent inference that the current throughput benchmarks cannot capture.
- Partner submissions: 19 partners submitted MLPerf v6.1 results, including Nebius, CoreWeave, Lambda, and Oracle Cloud Infrastructure. Eight submitted multi-node Blackwell NVL72 results. When partners start submitting Vera Rubin results independently, you will have a cross-validated performance picture rather than a single-vendor one.
- Software velocity: The 1.6x gain from v6.0 to v6.1 came from software alone. Track whether the same pace holds for Vera Rubin between its first and second MLPerf submission. If it does, the preview numbers are a floor, not a ceiling.
The binding constraint is now the power bill
NVIDIA is selling a thesis: that inference economics improve by an order of magnitude per generation, and that the improvement is concentrated in throughput per watt rather than raw FLOPS. The MLPerf v6.1 results support that thesis on traditional benchmarks. The AgentX preview extends it to agentic workloads, where the gains are larger because the workload pattern is harder and the architecture is better matched to it.
For builders, the practical question is whether your workload looks more like a fixed-input benchmark or an agent session with growing context, tool calls, and sub-agent spawning. If it is the latter, the efficiency gap between Vera Rubin and everything before it is wide enough to change your deployment math. If it is the former, the 3.7x MLPerf gain is still meaningful but remains an incremental step.
The infrastructure decision is now about how much intelligence you can produce per megawatt, and that number is moving faster than any cloud pricing model has caught up with.
Sources
- NVIDIA Blog: Vera Rubin NVL72 Delivers Leading Performance in MLPerf Inference v6.1 Debut
- NVIDIA Technical Blog: NVIDIA Vera Rubin and Blackwell Set a New Standard for Agentic AI Performance per Watt
- NVIDIA Blog: Vera Rubin NVL72 Sets a New Efficiency Standard for AI Agents
- NVIDIA Technical Blog: Inside the NVIDIA Vera Rubin Platform: Six New Chips, One AI Supercomputer
- NVIDIA Product Page: Vera Rubin NVL72
