Hibiki-Zero speech translation model eliminates need for aligned data

Researchers have developed Hibiki-Zero, a novel speech-to-speech translation system that bypasses the need for word-level alignment data. This approach simplifies training and allows for easier adaptation to various languages. The system first trains on sentence-level aligned data to achieve high latency translation, then uses a reinforcement learning strategy with GRPO to optimize for lower latency while maintaining translation quality. Hibiki-Zero has demonstrated state-of-the-art performance across five language pairs and can be adapted to new languages with minimal data. AI

IMPACT This model could significantly reduce the data requirements for developing speech translation systems, enabling broader language support and faster deployment.

RANK_REASON Research paper detailing a new model and method for speech translation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• GRPO
• Hibiki-Zero
• Hugging Face
• Tom Labiausse

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →