Dense 与 MoE 模型对比：活跃参数、吞吐量与如何选择
Dense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each
稠密模型与 MoE 架构的核心差异在于参数激活方式：MoE 通过专家路由让 30B 参数模型每个 token 仅激活 3B 参数，却仍能利用大模型容量，Nemotron 3.5 Lightning 即为例证。文章围绕活跃参数、吞吐量两个维度展开对比，并给出两者各自的适用选择场景。
How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the answer: It uses a Mixture-of-Experts (MoE) architecture that selects only a subset of its parameters for each token. There are two dominant model architectures: Dense model and MoE. How a model organizes its parameters matters as much as…
来源：NVIDIA Developer Blog · developer.nvidia.com