Miles C. Williams

New methods accelerate LLM inference with speculative decoding · 4 sources tracked

Researchers are developing new methods to accelerate large language model (LLM) inference through speculative decoding. AdaFlash, for instance, uses on-policy distillation and an adaptive length head to reduce variance …