Where is Megatron-LM used in practice?

Together with DeepSpeed's ZeRO and PyTorch's FSDP, Megatron-LM forms the trio of major large-scale training frameworks used by frontier labs and academic supercomputing centers. AI governance teams encounter Megatron-LM in the context of self-hosted LLM training projects, particularly at organizations with NVIDIA DGX clusters or substantial cloud GPU commitments. The platfo