Researchers have developed a new method called "Preserve to Expand" for adapting multilingual Automatic Speech Recognition (ASR) systems to new languages. This approach focuses on training a connector that maps speech representations into a frozen large language model, aiming to add new languages without degrading performance on existing ones. The study evaluated different strategies for updating and merging these connectors, finding that while some methods like target-only fine-tuning lead to significant forgetting of source languages, a combination of SSU-Wanda and TIES merging offers a better trade-off for single-target expansion. For multi-target expansion, update merging proved stable for source retention, though recursive expansion showed order sensitivity. AI
IMPACT This research could lead to more efficient development of multilingual ASR systems, reducing the need for extensive retraining and data.
RANK_REASON The cluster contains an academic paper detailing a new method for adapting LLM-based ASR systems. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Hugging Face
- Preserve to Expand: Source-Aware Adaptation and Model Merging for Multilingual LLM-Based ASR
- Speech LLMs
- SSU-Wanda+TIES
- TIES-Merging: Resolving Interference When Merging Models
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →