What is Chunked Prefill?

Chunked prefill addresses a throughput problem in inference serving: a very long prompt otherwise occupies the accelerator for an extended period before generation begins, blocking shorter requests behind it. Breaking the input into segments allows the work to interleave, which improves utilization and shortens the time other users wait. It is a serving optimization rather than a capability change — the model produces the same output either way. The governance relevance is indirect and worth naming honestly: efficiency improvements make longer prompts cheaper, which encourages sending more retrieved content, which increases the chance that retrieved text crowds out the instructions meant to govern it. Centralpoint addresses that with allocation rather than capacity. SkillTokenBudget fixes what each execution may spend before it runs, and governance skills are pre-indexed and loaded ahead of any retrieved material, so a larger retrieval set cannot displace the rules.