K-Forcing

K-Forcing accelerates LLM inference by decoding multiple tokens at once

Researchers have introduced K-Forcing, a new paradigm for accelerating language model inference by decoding multiple tokens simultaneously. This push-forward approach distills an existing autoregressive model into a map…