FP16 on Rik Kisnah - Bloghttps://www.rik-kisnah.ai/tags/fp16/Recent content in FP16 on Rik Kisnah - BlogHugoenTue, 15 Feb 2022 09:00:00 -0800Number Formats: FP32 to FP8https://www.rik-kisnah.ai/teach/gpu-ai/number-formats-fp32-to-fp8/Tue, 15 Feb 2022 09:00:00 -0800https://www.rik-kisnah.ai/teach/gpu-ai/number-formats-fp32-to-fp8/Every number in a model takes bits. Halve the bits and you double the speed, halve the memory, and risk the maths going wrong. Know which format keeps range, which keeps precision, and where you must not cut.