What is Prefix Caching?

Prefix caching is an LLM inference optimization that reuses the computed KV cache for shared prompt prefixes across multiple requests, eliminating redundant computation when many requests share the same system prompt, instructions, or document context. The platform strengthens enterpri