https://www.rik-kisnah.ai/tags/distributed-systems/Recent content in Distributed Systems on Rik Kisnah - BlogHugoenTue, 24 Jan 2023 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-a-distributed-job-scheduler/Tue, 24 Jan 2023 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-a-distributed-job-scheduler/A to-do list many workers pull from. Run each job once, on time, even when a worker dies half way through. The three words that make it work: lease, retry, idempotent.https://www.rik-kisnah.ai/teach/systems/design-an-object-store/Tue, 09 Nov 2021 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-an-object-store/Put a file in, get the same bytes back ten years later, from any of a hundred million files, after a hundred disks have died. Separate the names from the bytes, cut every file into pieces with spares, and check the pieces forever.https://www.rik-kisnah.ai/teach/systems/design-a-distributed-lock/Tue, 11 May 2021 09:00:00 -0700https://www.rik-kisnah.ai/teach/systems/design-a-distributed-lock/Only one process at a time may touch the thing. Easy on one machine. Across a network, the lock holder can die, pause, or believe it still holds the lock when it does not. Leases, fencing tokens, and the honest answer: a lock cannot make an unsafe operation safe.https://www.rik-kisnah.ai/teach/systems/design-a-unique-id-generator/Tue, 11 Feb 2020 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-a-unique-id-generator/Hand out numbers that are unique across a thousand machines, roughly in time order, without those machines ever talking to each other. Pack a timestamp, a machine id and a counter into 64 bits and you are done. Then the clock goes backwards.https://www.rik-kisnah.ai/teach/systems/consistent-hashing/Tue, 19 Mar 2019 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/consistent-hashing/Put the servers on a clock face. Put each key on the clock face too. Walk clockwise to find its server. Add a server and only the keys just behind it move. This is how caches, key-value stores and load balancers all share work without a reshuffle.https://www.rik-kisnah.ai/teach/systems/design-a-rate-limiter/Tue, 20 Mar 2018 09:00:00 -0700https://www.rik-kisnah.ai/teach/systems/design-a-rate-limiter/A jar of tickets that refills every second. Take one to enter. When the jar is empty, wait. Then the real questions: where does the jar live, and what happens when the jar breaks?https://www.rik-kisnah.ai/teach/systems/design-a-message-queue/Tue, 12 Sep 2017 09:00:00 -0700https://www.rik-kisnah.ai/teach/systems/design-a-message-queue/A post box between the person who writes letters and the person who reads them. The writer does not wait, the reader does not lose letters, and nobody has to be awake at the same time. Then: what if the post box burns down?https://www.rik-kisnah.ai/teach/systems/design-a-key-value-store/Tue, 07 Feb 2017 09:00:00 -0800https://www.rik-kisnah.ai/teach/systems/design-a-key-value-store/A giant dictionary spread across many machines. Put a value under a key, get it back later, even when machines die. Partition, replicate, and decide what ’later’ means when two copies disagree.