PulseAugur
LIVE 15:20:30
research · [2 sources] ·
0
research

New benchmark tests MLLMs' Chinese sign language understanding capabilities

By PulseAugur Editorial · Summary by gemini-2.5-flash-lite from 2 sources

Researchers have developed CNSL-bench, a new benchmark designed to evaluate the sign language understanding capabilities of multimodal large language models (MLLMs). This benchmark is grounded in the official Chinese National Sign Language Dictionary and includes aligned textual descriptions, images, and videos, covering diverse articulatory forms. Evaluations of 21 MLLMs using CNSL-bench revealed that current models significantly underperform human capabilities, with notable disparities across different input modalities and articulation types. AI

Summary written by gemini-2.5-flash-lite from 2 sources. How we write summaries →

IMPACT Establishes a new evaluation standard for MLLMs in sign language, highlighting current performance gaps compared to human understanding.

RANK_REASON Academic paper introducing a new benchmark for evaluating MLLMs on sign language understanding.

Read on arXiv cs.CL →

COVERAGE [2]

  1. arXiv cs.CL TIER_1 · Rui Zhao, Xuewen Zhong, Xiaoyun Zheng, Jinsong Su, Yidong Chen ·

    CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language

    arXiv:2604.22367v1 Announce Type: new Abstract: Sign language research has achieved significant progress due to the advances in large language models (LLMs). However, the intrinsic ability of LLMs to understand sign language, especially in multimodal contexts, remains underexplor…

  2. arXiv cs.CL TIER_1 · Yidong Chen ·

    CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language

    Sign language research has achieved significant progress due to the advances in large language models (LLMs). However, the intrinsic ability of LLMs to understand sign language, especially in multimodal contexts, remains underexplored. To address this limitation, we introduce CNS…

RELATED ENTITIES

RELATED TOPICS