用 NVIDIA Attribution 诊断分布式 AI 工作负载故障
Diagnose Distributed AI Workload Failures with NVIDIA Attribution
NVIDIA Attribution 可诊断分布式 AI 工作负载故障，从 GPU 或 fabric 事件引发的 NCCL 报错、调度器状态变化与海量下游消息中定位故障起点，而非工作负载停止的位置。文章演示了如何对样本工作负载证据运行 NVIDIA Attribution。
In distributed AI workloads, a single error can produce a cascade of symptoms. A GPU or fabric event may surface as NCCL errors on other ranks, scheduler state...
In distributed AI workloads, a single error can produce a cascade of symptoms. A GPU or fabric event may surface as NCCL errors on other ranks, scheduler state changes, and thousands of downstream messages. The reported exception shows where the workload stopped, not where the failure began. In this post, you’ll learn how to run NVIDIA Attribution on sample workload evidence…
来源：NVIDIA Developer Blog · developer.nvidia.com