Rik Kisnah - Blog

#GPU

Design the Training Network

Aug 15, 2023 Design the Training Network

Diagnose a Slow NCCL Job

Mar 14, 2023 Diagnose a Slow NCCL Job

Design a Topology-Aware GPU Scheduler

Oct 11, 2022 Design a Topology-Aware GPU Scheduler

All-Reduce Explained

May 10, 2022 All-Reduce Explained

Number Formats: FP32 to FP8

Feb 15, 2022 Number Formats: FP32 to FP8

Checkpointing at Scale

Dec 7, 2021 Checkpointing at Scale

Why GPUs for AI

Sep 14, 2021 Why GPUs for AI

GPU Faults: XID, ECC, and What They Mean

Apr 13, 2021 GPU Faults: XID, ECC, and What They Mean

Data, Model, and Pipeline Parallelism

Sep 15, 2020 Data, Model, and Pipeline Parallelism

RDMA Explained

Mar 10, 2020 RDMA Explained