What is ZeRO?

ZeRO, short for Zero Redundancy Optimizer, is a memory optimization technique introduced by Microsoft Research in 2019 that shards optimizer states, gradients, and (optionally) model parameters across data-parallel GPU ranks, eliminating the memory redundancy of traditional data parallelism. The platform strengthens enterprise readine