Gumbel-BEARD: Automatic Layer Selection for Self-Supervised Adaptation of Whisper in Low-Resource Domains
Researchers have developed Gumbel-BEARD, a novel framework designed to improve the performance of speech foundation models in low-resource domains. This method automates the selection of Whisper encoder layers using a trainable Gumbel-Softmax selector and a self-supervised adaptation objective. Experiments show that Gumbel-BEARD can match fully supervised baselines with significantly less labeled data and establishes new state-of-the-art word error rates on challenging datasets like MyST and CORAAL. AI
IMPACT Enhances speech model performance in low-resource settings, potentially broadening AI accessibility for diverse linguistic communities.