KnapSpec

New methods accelerate LLM inference with speculative decoding

Researchers have developed several new methods to accelerate large language model (LLM) inference through speculative decoding. AdaPLD improves retrieval and draft construction by using semantic similarity and branched …