How does Learning Rate work?

Modern LLM training uses learning rate schedules rather than constant values: warmup from zero to a peak rate over the first few thousand steps, then decay (cosine, linear, or constant) toward zero. Typical peak learning rates for LLM pretraining are 1e-4 to 3e-4, while fine-tuning uses lower values like 1e-5 to 1e-4 to avoid disrupting pretrained representations. The platform strengthens ente