Lukas Hauzenberger

xLSTM models achieve near-lossless distillation from larger LLMs

Researchers have developed an effective distillation pipeline to transfer knowledge from large language models (LLMs) with quadratic attention to sub-quadratic architectures based on xLSTM. This method aims for lossless…