How does Positional Encoding work?

The original 2017 Transformer paper used fixed sinusoidal positional encodings added to input embeddings. Modern LLMs use more sophisticated approaches: learned absolute positional embeddings (early GPT models), RoPE (most current models including Llama, Mistral, Qwen, Gemma), ALiBi (used in BLOOM and MPT), and various hybrid approaches. The platform strengthen