New Transformer Model Achieves SOTA in Sign Language Recognition

Researchers have developed a novel Transformer-based architecture called the Sequential Spatio-Temporal Attention Network (SSTAN) for dynamic sign language and fingerspelling recognition. This model utilizes hierarchical, stacked Spatial and Temporal Multi-Head Attention mechanisms to capture complex spatio-temporal patterns without relying on predefined graph structures. Experiments on large-scale datasets like WLASL, JSL, and KSL demonstrated that SSTAN achieves state-of-the-art performance, particularly in challenging fingerspelling categories, and establishes a new SOTA for skeleton-only methods on WLASL, showcasing its data efficiency. AI

IMPACT This research advances sign language recognition capabilities, potentially improving communication tools for the deaf community.

RANK_REASON The cluster contains an academic paper detailing a new model architecture and its performance on specific benchmarks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• Japanese Sign Language
• Koki Hirooka
• KSL-TV
• Sequential Spatio-Temporal Attention Network
• Spatial Multi-Head Attention
• SSTAN
• Stack Transformer
• Temporal MHA
• WLASL

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →