How does FSDP work?

Unlike traditional data parallelism that replicates the full model on each GPU, FSDP keeps only a shard on each device and gathers full parameters just-in-time during forward and backward passes. FSDP is conceptually equivalent to DeepSpeed's ZeRO stage 3 but implemented natively in PyTorch with cleaner integration. The platform strengthens enterprise re