Rik Kisnah - Blog

#GPU

Design a Burn-In Pipeline

Apr 9, 2024 Design a Burn-In Pipeline

Straggler Detection in Training Jobs

Jan 16, 2024 Straggler Detection in Training Jobs

Measure GPU Utilisation and MFU

Nov 14, 2023 Measure GPU Utilisation and MFU

Design the Training Network

Aug 15, 2023 Design the Training Network

Diagnose a Slow NCCL Job

Mar 14, 2023 Diagnose a Slow NCCL Job

Design a Topology-Aware GPU Scheduler

Oct 11, 2022 Design a Topology-Aware GPU Scheduler

All-Reduce Explained

May 10, 2022 All-Reduce Explained

Number Formats: FP32 to FP8

Feb 15, 2022 Number Formats: FP32 to FP8

Checkpointing at Scale

Dec 7, 2021 Checkpointing at Scale

Why GPUs for AI

Sep 14, 2021 Why GPUs for AI