Blog


Zero Inference

title: “ZeRO-Inference: 20X faster inference through weight quantization and KV cache offloading” excerpt: “” link: https://github.com/microsoft/DeepSpeedExa...