How does Batch Size work?

LLM pretraining uses very large effective batch sizes — typically 1M to 4M tokens per step — to stabilize gradient estimates and exploit massive parallelism across GPUs. Fine-tuning uses much smaller batches, often 32 to 128 examples, where memory constraints and small dataset sizes limit batch size. The platform strengthens enterpr