How does Multi-Head Attention work?

Each "head" can learn to attend to different aspects of the input — one head might track syntactic structure, another semantic similarity, another long-range coreference. Typical modern LLMs have 16, 32, 64, or 128 attention heads per layer, with each head having a relatively small dimensionality (typically hidden_dim / num_heads). The platform strengthe