Cross-Modal Masking for Robust Silent Speech Synthesis Using sEMG and Lipreading
Researchers have developed a new framework for silent speech synthesis that combines surface electromyography (sEMG) and lipreading data. This approach uses modality masking during training to improve robustness against sensor failure or signal degradation. The masked multimodal system significantly reduced word error rates compared to unimodal methods, particularly for vowels and certain consonant groups, demonstrating its effectiveness for assistive technology. AI
IMPACT This research advances assistive technologies by improving the robustness and accuracy of silent speech synthesis systems.