FEA-SLT: A Gloss-Free End-to-End Framework for Facial-Expression-Aware Sign Language Translation
Researchers are exploring novel methods to improve sign language translation (SLT) by leveraging large language models and advanced encoding techniques. One approach uses GPT-4o to generate paraphrased target sentences, augmenting training data to enhance translation quality, particularly for languages with sparse vocabularies. Another method, FEA-SLT, integrates facial expressions as semantic anchors to resolve ambiguities in manual sign configurations, achieving state-of-the-art results among gloss-free methods. Additionally, the SAGE framework introduces segment-aware visual tokenization to create more efficient and scalable gloss-free SLT models by reducing input sequence lengths. AI
IMPACT Advances in LLM integration and efficient encoding promise more accurate and scalable sign language translation systems.