Why does Speculative Decoding matter for AI governance?

The technique requires a draft model from the same family as the target model — Llama 3.1 8B as draft for Llama 3.1 70B, for example. vLLM, TensorRT-LLM, and DeepMind's reference implementation all support speculative decoding. Th