https://www.rik-kisnah.ai/tags/performance/Recent content in Performance on Rik Kisnah - BlogHugoenTue, 14 Nov 2023 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/measure-gpu-utilisation-and-mfu/Tue, 14 Nov 2023 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/measure-gpu-utilisation-and-mfu/‘GPU utilisation 100 percent’ can mean the chip is doing useful maths, or that one thread is spinning. Model FLOPs utilisation is the honest number: useful work divided by what the chip could do. Know how to compute it and what a good value is.https://www.rik-kisnah.ai/teach/gpu-ai/diagnose-a-slow-nccl-job/Tue, 14 Mar 2023 09:00:00 -0700https://www.rik-kisnah.ai/teach/gpu-ai/diagnose-a-slow-nccl-job/The job runs. Nothing errors. It is just half as fast as last week. This is the ticket every GPU infrastructure engineer gets, and it has a fixed order of questions. Learn the order and you solve it in an hour instead of a week.https://www.rik-kisnah.ai/teach/gpu-ai/number-formats-fp32-to-fp8/Tue, 15 Feb 2022 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/number-formats-fp32-to-fp8/Every number in a model takes bits. Halve the bits and you double the speed, halve the memory, and risk the maths going wrong. Know which format keeps range, which keeps precision, and where you must not cut.https://www.rik-kisnah.ai/teach/gpu-ai/gpu-memory-hierarchy/Tue, 12 Nov 2019 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/gpu-memory-hierarchy/Registers, shared memory, HBM, then the CPU across the bus. Each level is bigger and slower than the last by a lot. Most slow GPU code is fast GPU code waiting for the wrong level.https://www.rik-kisnah.ai/teach/systems/design-a-distributed-cache/Tue, 10 Jul 2018 09:00:00 -0700https://www.rik-kisnah.ai/teach/systems/design-a-distributed-cache/A shelf by the door where you keep the things you reach for every day. Fast to grab, small, and dangerous when what is on the shelf is no longer true. Eviction, invalidation, and the stampede.