benchmarkNVIDIA

NVIDIA Vera Rubin NVL72 posts over 10x GB300 NVL72 throughput per kW on MLPerf Endpoints v1.0

TL;DR

Provisional MLPerf Endpoints v1.0 results show NVIDIA Vera Rubin NVL72 delivering over 10x the throughput per kilowatt of GB300 NVL72 at a shared minimum interactivity of 50 tokens per second per user. The Vera Rubin results are pending MLCommons verification; the GB300 results are verified.

3 min read
0

NVIDIA's Vera Rubin NVL72 delivers over 10x the throughput per kilowatt of provisioned power of GB300 NVL72 in provisional MLPerf Endpoints v1.0 results, measured at a shared minimum end-to-end (E2E) interactivity of 50 tokens per second per user. NVIDIA published the analysis on its developer blog on November 16, 2026. The Vera Rubin results are pending MLCommons Association verification and are subject to change pending peer review. The GB300 results are verified by MLCommons Association.

What MLPerf Endpoints v1.0 measures

The benchmark evaluates the balance between serving capacity and responsiveness using coding and enterprise workflows through API endpoints. During performance runs, generated tool calls and workflow intent are checked against reference values. Separate SWE-bench Verified evaluations assess end-to-end task-solving capability, to test whether faster serving preserves model quality.

According to NVIDIA, the dataset reflects multi-turn agentic behavior: coding trajectories accumulate history over many turns, while enterprise workflows reuse shared instructions, tool definitions and business rules. In NVIDIA's Figure 1, covering 230 coding and workflow trajectories with more than 10 model responses, median context length rises from about 30,000 tokens at the first response to about 200,000 tokens by response 300.

Results by model

The submissions cover Kimi K3, DeepSeek-V4.1-Flash and Qwen3.6-35B-A3B (Standardized division, Available category). NVIDIA's comparison of Vera Rubin NVL72 against GB300 NVL72:

Model Throughput per provisioned kW ratio E2E interactivity ratio
Kimi K3 10x 3x
DeepSeek-V4.1-Flash 9x 5x
Qwen3.6-35B-A3B 5x 3x

The interactivity ratios are at matched query concurrency. Each platform's best TPS/kW meeting the stated minimum interactivity was selected for the throughput comparison.

Kimi K3 operating points

NVIDIA's footnote gives the underlying records, retrieved from endpoints.mlcommons.org on November 9, 2026:

  • Vera Rubin NVL72, 32 accelerators: concurrency of 1,024 queries, system throughput of 41,414.2 tokens/sec, E2E average interactivity of 50.4 tokens/sec per user, operating power of 100.89 kW.
  • GB300 NVL72, 48 accelerators: concurrency of 48 queries, system throughput of 2,210.4 tokens/sec, E2E average interactivity of 56.2 tokens/sec per user, operating power of 60 kW.

GB300 results were published November 4, 2026. Vera Rubin results were published November 9, 2026 for Kimi K3 and Qwen, and November 10, 2026 for DeepSeek.

Hardware and software behind the results

NVIDIA attributes the results to Rubin GPUs, which combine HBM4 memory, Tensor Cores and sixth-generation NVLink. On the software side, NVIDIA cites context-aware serving techniques: prefix caching, state reuse and NVIDIA Dynamo routing. These reduce redundant computation and balance load across agentic workloads.

NVIDIA also lists model-specific execution optimizations. DeepSeek's MegaMoE kernel and Qwen's multi-token prediction with Tensor Core verification reduce data movement and sequential decoding steps.

What this means

The headline 10x figure comes from one model (Kimi K3) at one interactivity threshold, 50 tokens per second per user. The other two models show 9x (DeepSeek-V4.1-Flash) and 5x (Qwen3.6-35B-A3B) on throughput per kilowatt. Teams sizing agentic deployments should read the per-model ratios rather than the top-line number.

The verification status also matters. Only the GB300 baseline is verified by MLCommons Association; the Vera Rubin numbers may change. The two systems were also compared at different accelerator counts (32 for Vera Rubin, 48 for GB300) and different concurrency levels at their selected Kimi K3 operating points, so the per-kilowatt metric, not per-accelerator performance, is the basis of the comparison.

Because the benchmark pairs performance runs with tool-call and intent checks against reference values, plus separate SWE-bench Verified evaluations, the results are meant to show throughput gains alongside a check on output quality. The full published data is available at MLPerf Endpoints results.

Related Articles

benchmark

NVIDIA Vera Rubin NVL72 posts fastest time to train on six of seven MLPerf Training v6.1 benchmarks

In its first MLPerf Training appearance, NVIDIA's Vera Rubin NVL72 delivered the fastest time to train on six of seven benchmarks, with up to 2x the per-GPU performance of an optimized GB300 NVL72 baseline, according to NVIDIA. On DeepSeek-V3 671B it trained in 4.14 minutes on 1,024 GPUs versus 8.03 minutes for GB300 NVL72.

product update

NVIDIA launches Nemotron for Science with Nemotron 3.5 Super VL and a chemistry dataset of nearly 50,000 questions

NVIDIA introduced Nemotron for Science on November 16, 2026. It pairs the open Nemotron 3.5 Super VL model with scientific evaluations, post-training recipes and LitmusBench v0.2, a chemistry dataset of nearly 50,000 questions.

research

NVIDIA recipe: RL post-training lifts Nemotron 3.5 Super VL accuracy a further 17% on drug-metabolism task

NVIDIA has published a recipe for post-training Nemotron 3.5 Super VL on scientific tasks using NeMo Gym and NeMo RL. In a drug-metabolism example, supervised fine-tuning improved accuracy by 12% and RL post-training lifted it a further 17% at a 16,384-token output budget.

benchmark

NVIDIA serves 753B-parameter GLM-5.3 NVFP4 on two DGX Stations, peaking at 607.8 tokens/s

NVIDIA published a tutorial for serving GLM-5.3 NVFP4, a 753-billion-parameter MoE model, across two GB300 DGX Stations with vLLM. In NVIDIA's AIPerf tests, output throughput peaked at 607.8 tokens per second at concurrency 16 and fell to 385.0 at concurrency 32.

Comments

Loading...