NVIDIA GB300 Posts 20x Agentic AI Lead Over H200 In New AA AgentPerf Benchmark
NVIDIA’s Blackwell Ultra GB300 NVL72 has taken an early lead in agentic AI infrastructure, running up to 20x more concurrent agents per megawatt than H200 in a new benchmark from Artificial Analysis.
NVIDIA has published its first results for AA AgentPerf, a new Artificial Analysis benchmark designed to measure how AI systems handle real agentic workloads. As highlighted by Wccftech, the benchmark tests multi turn coding agents, long context requests, tool calls, variable sequence lengths, and sustained concurrent load. This matters because agentic AI is different from standard chatbot inference. A normal LLM request is usually a single response. An AI agent can chain dozens or hundreds of model calls, tool calls, code edits, and tests before finishing a task. That makes concurrency, scheduler behavior, KV cache reuse, and output speed much more important.
According to NVIDIA, GB300 NVL72 delivered up to 61.4K concurrent agents per MW using DeepSeek V4 Pro, compared with 2.6K for H200. Per GPU, GB300 reached 57.5 concurrent agents versus 1.4 for H200.
| Benchmark | Value of metric | NVIDIA GB300 NVL72 | NVIDIA H200 |
|---|---|---|---|
| Concurrent agents per MW | Energy efficiency, active agents supported for a given power budget | 61.4K | 2.6K |
| Concurrent agents per GPU | Hardware efficiency, serving capacity achieved per GPU | 57.5 | 1.4 |
NVIDIA says the result comes from full stack optimization across Blackwell Ultra, NVLink, inference software, and mixture of experts execution. GB300 NVL72 connects 72 GPUs inside a single rack scale system, helping large models distribute workloads more efficiently across many concurrent agent sessions.
AA AgentPerf also focuses on real deployment behavior instead of synthetic prompts. It measures time to first token, output speed, and total system output throughput while maintaining service level targets. That makes the benchmark more relevant for companies planning agentic coding platforms, internal developer agents, AI copilots, and production inference services.
This also connects with NVIDIA’s Quantum X CPO switch at Lambda. As AI agents create more traffic across clusters, rack scale compute, networking efficiency, and power allocation are becoming one combined infrastructure problem.
NVIDIA is already pointing toward Vera Rubin as the next major step. The platform is expected to push performance further with NVFP4 compute, Vera CPU integration, and stronger end to end efficiency for workloads that mix model inference, tool calls, and long running agent sessions. That is important because agentic AI changes what matters in data centers. Raw FLOPS still matter, but the real business question is how many useful agents can run at once within a fixed power and cost envelope.
GB300’s 20x lead over H200 is not just a speed story. It is a power efficiency story.
AI factories are increasingly limited by power, cooling, networking, and total cost of ownership. If one rack scale system can support far more active agents per MW, that directly affects how cloud providers price agentic AI and how enterprises decide what hardware to deploy.
The caution is that AA AgentPerf is still new. It is useful because it reflects real agentic coding behavior, but the industry will need more model coverage, more vendor results, and more independent validation before treating it like a mature standard.
Even so, the direction is clear. The AI hardware battle is moving from single request benchmarks to agent capacity at scale. On that front, GB300 is giving NVIDIA another strong lead before Rubin arrives.
Do you think agentic AI benchmarks like AA AgentPerf will become more important than traditional inference benchmarks?
