NVIDIA Dynamo-Triton 集成 TensorRT 多设备推理，支持单网络跨多 GPU 执行
Simplifying Model Serving Across Multiple GPUs with NVIDIA TensorRT Multi-Device Integration in NVIDIA Dynamo-Triton
NVIDIA TensorRT 多设备推理从 TensorRT 11.0 起正式支持，让单个 TensorRT 网络借助 NCCL 分布式集合通信跨多块 GPU 执行，同时保留 TensorRT 的推理优化。该能力已集成进 NVIDIA Dynamo-Triton，用于应对生成式 AI 超出单 GPU 的算力与显存需求。
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability...
The compute and memory demands of generative AI increasingly exceed what a single GPU can provide. NVIDIA TensorRT multi-device inference is a new capability that enables a single TensorRT network to execute across multiple GPUs using NCCL-backed distributed collectives while retaining TensorRT inference optimizations. It is fully supported starting with TensorRT 11.0.
来源：NVIDIA Developer Blog · developer.nvidia.com