How does Speculative Decoding work?

The draft model speculatively generates a sequence of K tokens (typically 4-8), and the target model verifies them all in a single forward pass — accepting the longest prefix that matches what the target model would have produced. Successful speculation produces multiple tokens per target-model step, accelerating inference by 2x-3x with no quality loss because the output distribution exactly matches the target model's. The platform strengthe