Brief

Researchers have developed ZeSTA, a new framework for improving personalized speech synthesis using zero-shot text-to-speech (ZS-TTS) as a data augmentation source. The method addresses the common issue of speaker similarity degradation when mixing synthetic and real speech data during fine-tuning. ZeSTA employs a domain-conditioned training approach that distinguishes between real and synthetic speech, coupled with oversampling of real data to stabilize adaptation, particularly in low-resource scenarios. AI

A new research paper analyzes how errors in Korean speech recognition impact the performance of large language models (LLMs) in spoken question answering (SQA). The study found that the degradation caused by speech recognition errors is consistent across different LLMs, suggesting that the information loss at the speech recognition stage is the primary driver of performance decline. The research also identified single-character errors in Korean transcriptions as a unique vulnerability that can alter the intended question and degrade QA accuracy. An auxiliary comparison indicated that large audio language models may offer a more robust solution by directly processing audio input, potentially mitigating issues caused by transcription errors. AI