New gradient-based method aligns speech-to-text across all ASR models

Researchers have developed a novel gradient-based method for aligning speech-to-text, applicable to any differentiable automatic speech recognition (ASR) model. This technique derives word timings from the gradient of token log probabilities, bypassing the need for model modification or training. It offers a generic alignment solution that works across various ASR families, including speech LLMs, and aligns directly on the input grid for greater temporal precision. Evaluations on sixteen models across read and spontaneous speech datasets show the gradient alignment is usable, performing comparably to native aligners and offering improvements for streaming models, though it requires a backward pass per token. AI

IMPACT This new alignment technique could improve the accuracy and flexibility of speech-to-text systems across various model architectures.

RANK_REASON The cluster describes a new research paper detailing a novel method for speech-to-text alignment.

Read on arXiv cs.CL →

• arXiv
• ASR model
• Buckeye
• Hugging Face
• Speech LLMs
• TIMIT
• alphaXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →