Capacity Planning on Rik Kisnah - Bloghttps://www.rik-kisnah.ai/tags/capacity-planning/Recent content in Capacity Planning on Rik Kisnah - BlogHugoenTue, 11 Nov 2025 09:00:00 -0800Design an Inference APIhttps://www.rik-kisnah.ai/teach/gpu-ai/design-an-inference-api/Tue, 11 Nov 2025 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/design-an-inference-api/Thousands of people send prompts, a pool of GPUs answers them, and the GPU only earns its keep when it works on many prompts at once. Design the layer in between: the batcher, the queues, the maths for how many GPUs, what happens when one dies, and how two models share eight GPUs when one of them needs all eight.