https://www.rik-kisnah.ai/tags/scheduling/Recent content in Scheduling on Rik Kisnah - BlogHugoenTue, 11 Nov 2025 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/design-an-inference-api/Tue, 11 Nov 2025 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/design-an-inference-api/Thousands of people send prompts, a pool of GPUs answers them, and the GPU only earns its keep when it works on many prompts at once. Design the layer in between: the batcher, the queues, the maths for how many GPUs, what happens when one dies, and how two models share eight GPUs when one of them needs all eight.https://www.rik-kisnah.ai/teach/systems/design-a-distributed-job-scheduler/Tue, 24 Jan 2023 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-a-distributed-job-scheduler/A to-do list many workers pull from. Run each job once, on time, even when a worker dies half way through. The three words that make it work: lease, retry, idempotent.https://www.rik-kisnah.ai/teach/gpu-ai/design-a-topology-aware-gpu-scheduler/Tue, 11 Oct 2022 09:00:00 -0700https://www.rik-kisnah.ai/teach/gpu-ai/design-a-topology-aware-gpu-scheduler/Which GPUs a job lands on decides how fast it runs. A scheduler that hands out any free GPU is a scheduler that puts one job’s ring across three spines. Give it a map of the cluster and make it place jobs the way the network wants.