LogitSpec

New methods accelerate LLM inference via speculative decoding improvements

Researchers are developing new methods to accelerate large language model (LLM) inference, a process often slowed by sequential decoding. Several recent papers explore speculative decoding techniques that use a smaller …