ASKD-Whisper: Adaptive Self-knowledge Distillation for Efficient and Low-Latency Automatic Speech Recognition
Researchers have developed Adaptive Self-Knowledge Distillation (ASKD), a novel framework for compressing large AI models. This method dynamically reduces reliance on a teacher model's predictions during training, encouraging the student model to develop independent reasoning. ASKD was applied to distill the Whisper speech recognition model into a more efficient version, ASKD-Whisper, which achieved a 5x reduction in inference latency and a 1.07% lower word error rate compared to its teacher. AI
IMPACT This technique could enable more efficient deployment of large ASR models on resource-constrained devices.