NVIDIA GB300 NVL72 创下 MoE 预训练世界纪录
Setting a World Record for MoE Pre-Training on NVIDIA GB300 NVL72
NVIDIA GB300 NVL72 在预训练 DeepSeek-V3 671B 时创下每 GPU 1,648 TFLOPs 的世界纪录。随着前沿模型预训练转向 MoE 架构，每 token 计算量下降，通信效率日益决定模型在数千块 GPU 上的扩展表现。
Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token...
Frontier model pre-training has converged on mixture of experts (MoE), which is fundamentally changing what limits large-scale AI training. As compute per token falls, communication increasingly determines how efficiently models scale across thousands of GPUs. NVIDIA GB300 NVL72 set a world record for pre-training DeepSeek-V3 671B at 1,648 TFLOPs per GPU, showing how advances across the entire AI…
来源：NVIDIA Developer Blog · developer.nvidia.com