Top 100 · Inference Optimization49 of 100

Identify the B200 and GB200 NVL72 in NVIDIA's Blackwell lineup.

Flashcard·Easy·4.0 · 0·~30s

Attempt it

Your answer

The **B200** is NVIDIA's Blackwell-generation datacenter GPU, the successor to the Hopper H100 and H200. Released in 2024 and shipping in volume through 2025-2026, the B200 has 192 GB of HBM3e memory at roughly 8 TB/s of bandwidth, and its tensor cores support a wider range of low-precision formats than Hopper: native FP8 (carried over from Hopper) plus native FP4. The **GB200 NVL72** is a rack-scale system, not a single GPU. It packages 72 B200 GPUs into one rack, all connected by an NVLink switch fabric so any GPU can address any other GPU's HBM at NVLink bandwidth (much faster than networking between separate servers). The 72 GPUs together expose more than 13 TB of HBM3e on a single NVLink domain. Why this matters for 2026 inference. **The B200's FP4 path** halves bytes per weight again over FP8, which compounds quantization wins for both bandwidth and compute. **The GB200's shared fabric** makes it practical to serve models whose weights and KV cache exceed any single GPU's memory: large mixture of experts models like Llama 4 Maverick, very long context reasoning workloads, and multi-stage agent pipelines where tight all-to-all bandwidth would otherwise bottleneck. In 2026 the GB200 NVL72 is the de facto frontier inference platform. Anthropic, OpenAI, Google, and DeepSeek all use it (or comparable racks) for their highest-tier serving, and open-source stacks like vLLM, SGLang, and TensorRT-LLM have explicit support for the GB200 fabric model.

The **B200** is NVIDIA's Blackwell-generation datacenter GPU, the direct successor to H100 and H200. It ships with **192 GB of HBM3e memory** at roughly 8 TB/s of memory bandwidth, and its tensor cores support **native FP4 and FP8** in addition to BF16 and INT8. FP4 is the Blackwell-specific addition, doubling the low-precision compute path over Hopper's FP8 ceiling. The **GB200 NVL72** is a rack-scale system. It bundles 72 B200 GPUs (paired with 36 Grace CPUs) into a single chassis and links every GPU into one NVLink switch fabric. The fabric exposes a flat memory-addressable domain: any GPU can read any other GPU's HBM at NVLink bandwidth (roughly 1.8 TB/s per GPU on Blackwell's NVLink 5). Total HBM in the domain is over 13 TB at multi-PB/s aggregate bandwidth. Why this matters in 2026: **Single-GPU wins.** B200's 192 GB HBM3e is large enough to fit most frontier dense models in FP8 without tensor parallelism, and the FP4 path doubles compute for prefill-heavy workloads. FP4 KV cache on B200 cuts decode-time bandwidth pressure further compared to FP8 on Hopper. **Rack-scale wins.** Large mixture of experts models (Llama 4 Maverick, DeepSeek V4, Qwen 3 family at the largest tiers) split experts across many GPUs and rely on fast all-to-all communication between experts every token. On separate servers connected by Ethernet or InfiniBand, that all-to-all is the bottleneck. On NVL72's shared NVLink, it runs at HBM-like speeds. The same logic applies to extremely long context windows where KV cache exceeds any single GPU's memory and needs to be partitioned across the fabric. In production, the GB200 NVL72 has become the reference platform for frontier serving in 2026. The major labs use it (or their hyperscaler equivalents) for their top-tier model deployments, and the open-source serving stacks support its fabric model explicitly.

Quick reference

TL;DR

B200 is NVIDIA's Blackwell-generation single GPU: 192 GB HBM3e and native FP4 plus FP8 tensor cores.

Memory aid

Easy to grasp

Think about scaling up a kitchen. A bigger oven (the B200) holds more food, cooks at lower temperatures the older oven could not handle, and finishes faster. But the kitchen still has space limits. A rack-scale system like GB200 NVL72 is more like converting a whole floor into one shared kitchen with 72 connected ovens and one giant pantry everyone can reach. Now you can prepare a banquet that no single oven could have managed alone, because every cook has direct access to every ingredient. For language model serving, the bigger oven is helpful, but the connected-kitchen layout is what makes serving a trillion-parameter mixture of experts or a multi-rack reasoning workload practical without sending data over slow connections.

Key concepts

Inference hardware basics NVIDIA B200 NVLink fabric

Detailed answer & concept explanation

~6 min read

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example. Click to expand.

Everything you need to truly understand this topic: intuition, mechanics, step by step explanation, code, formulas, and worked example.

Quick scan · 2–4 min

Everything important, quickly.

2 min: B200 single-GPU specs and FP4 path + GB200 NVL72 fabric model + why MoE and long-context workloads need it + cost framing and when it is the wrong fit.

Real-world examples

Real products, models, and research that use this idea.

Anthropic's frontier serving fleet for Claude Opus 4.7 uses GB200 NVL72-class hardware (or hyperscaler equivalents) for the largest reasoning workloads.
OpenAI's GPT-5.5 serving relies on NVL72-class fabric for routing the largest mixture of experts experts across many GPUs.
Meta's Llama 4 Maverick (MoE) inference at scale ships on GB200 NVL72 at Meta's own datacenters and at hyperscaler rentals.
DeepSeek V4's serving stack assumes NVL72-class fabric for its expert routing and long-context reasoning paths.
vLLM, SGLang, and TensorRT-LLM all added explicit GB200 NVL72 support paths in 2025-2026 because the open-source community needs to target the same hardware as the closed-source frontier.

Interviewer follow-ups

What an interviewer would ask next. Try answering before peeking at the approach.

QHow does FP4 on B200 change the roofline picture compared to FP8 on H100?

QWhat is the practical difference between NVL72's NVLink fabric and a multi-node H100 cluster connected by InfiniBand?

QWhen does NVL72 lose to a comparable number of disaggregated H100s on price-performance?

Don't say this

Red flags and common mistakes that signal junior thinking. Click to expand.

Most common mistake

Treating GB200 NVL72 as just 72 separate B200s. The point is the shared NVLink fabric that lets all 72 GPUs address each other's HBM at NVLink speeds, making the rack behave like one giant GPU.

60-second cheat sheet

The night-before-the-interview bullets. Scan these on the way to the call.

References

Primary sources. Skim if you want the original framing.