Scalable compute with Blackwell architecture
NVIDIA B200 chips contain 208 billion transistors and use TSMC 4NP process technology. The GB200 superchip combines two B200 GPUs with one Grace CPU to reach 40 petaflops of compute power. This configuration delivers 30x more inference model performance than the H100 while using 1/25 of the original energy and cost. The Blackwell architecture uses a 10 TB/s chip-to-chip interconnect to link two dies. For larger models, the Blackwell Ultra (GB300) increases HMB3e memory per GPU from 192GB to 288GB. The HGX B200 server is based on eight B200 GPUs and one x86 CPU to deliver 18 petaflops of FP4 throughput. Each B200 in this setup has a TDP of 1000W. The HGX B100 is a compatible option for existing HGX H100 infrastructure with 14 petaflops of FP4 per GPU and 700W TDP. Every Blackwell GPU provides 8 TB/s of HMB3e bandwidth. The B200 is two full reticle-sized chips, each surrounded by four HMB3e stacks. The GB200 superchip uses the NVLink-C2C interconnect to deliver 900 GB/s of bidirectional bandwidth between the two Blackwell GPUs and the Grace CPU. The GB200 NVL72 has a configuration of 72 GPUs in one rack or 72 GPUs in two racks.
| Specification | B200 | B100 | GB200 NVL72 |
|---|---|---|---|
| FP4 Compute | 20 PFLOPS | 14 PFLOPS | 720 PFLOPS |
| FP8/FP6 Compute | 10 PFLOPS | 7 PFLOPS | 720 PFLOPS |
| Memory | 192 GB HMB3e | 192 GB HMB3e | 13.4 TB HMB3e |
| Bandwidth | 8 TB/s | 8 TB/s | 576 TB/s |
Production delays and supply limits
A design flaw in the processor die that connects two Blackwell GPUs on a single superchip forced a three-month delay in production. This flaw appeared during the production phase at TSMC. The limited supply of CoWoS-L packaging technology also complicates mass production. You should know that supply chains for HMB3e memory and TSMC capacity remain strained through 2026. SK Hynix and Micron filled their 2025 capacity entirely. The growing demand for AI infrastructure outpaced the global supply chain’s ability to respond. While OEM issues with liquid-cooled racks were resolved, the first two links in the chain remain bottlenecks. Demand for the B200 and GB200 NVL72 racks exceeded supply through mid-2026. The industry anticipates that the Rubin architecture will enter the market in the second half of 2026.
Inference and training speeds
The GB200 NVL72 connects 36 Grace CPUs and 72 Blackwell GPUs in a liquid-cooled rack-scale design to deliver 30x faster real-time trillion-parameter large language model inference performance compared to the previous H100 generation. This system provides 130 TB/s of low-latency GPU communications for AI workloads. The fifth-generation NVLink provides 1.8 TB/s of GPU-to-GPU interconnect. This interconnect helps overcome communication bottlenecks in large clusters. The GB200 NVL72 also uses a second-generation Transformer Engine. This engine enables FP4 AI. The second-generation Transformer Engine enables FP8 precision for 4x faster training for large language models at scale. The 1.8T parameter GPT-MoE achieves 30x speedup compared to the H100 generation. The GB200 NVL72 provides 720 PFLOPS of FP8/FP6 compute and 360 PFLOPS of FP16/BF16 compute. The system contains 2,592 Arm Neoverse V2 cores and 17 TB of LPDDR5X memory. The GB200 compute tray provides 80 petaflops of AI performance and 1.7 TB of fast memory. The dedicated decompression engine supports LZ4, Deflate, and Snappy formats to speed up analytics. This engine allows Blackwell to perform 18x faster than CPUs for database queries. Additionally, the GB200 predicts Cadence SpectreX circuit simulations 13x quicker than traditional CPUs and Cadence Fidelity CFD simulations 22x faster. AMD’s MI355X has 288GB HMB3e, which exceeds the 270GB seen in the B300. Can companies overcome these physical supply bottlenecks before the next training cycle begins?
