What is FSDP?

FSDP, short for Fully Sharded Data Parallel, is a distributed training technique built into PyTorch since version 1.11 (2022) that shards model parameters, gradients, and optimizer states across multiple GPUs, dramatically reducing per-GPU memory and enabling training of very large models on commodity clusters. The platform strengthens enterprise readine