bonus logits

New speculative decoding methods boost LLM inference speed · 7 sources tracked

Researchers are advancing speculative decoding techniques for large language models to improve inference speed. Two new arXiv papers, ECHO and LoopSpec, introduce hierarchical and pipelined approaches, respectively, to …