What is ALiBi?

Attention with Linear Biases replaces learned positional embeddings with a distance-based penalty applied directly to attention scores. The practical benefit is extrapolation: a model trained on shorter sequences handles longer ones at inference without the degradation that learned embeddings exhibit outside their trained range. It appears in several notable open models and is one of a family of approaches to the same problem. For an organization governing AI, context length is worth understanding for a narrower reason than it is usually discussed — a longer window means more retrieved material competes with the instructions meant to govern how that material is used. Centralpoint addresses that with budgeting rather than with capacity: SkillTokenBudget fixes the allocation before a request runs, and pre-indexed governance skills load ahead of any retrieved content, so a large retrieval set cannot displace the rules.