Researchers have introduced MTG-AD, a new dataset and evaluation framework designed to assess the performance of multimodal large language models (MLLMs) in generating inline audio descriptions for videos. The MTG-AD dataset comprises 36 hours of professionally annotated videos, focusing on user-generated content. The accompanying MTG-AD-Eval framework provides fine-grained, reference-free metrics that evaluate the end-to-end task, including rendering scripts into audio to check for disruptive overlaps with existing video content. While current MLLMs show promise in reducing speech overlaps and describing visual events, they still face challenges in narrative pacing and flow, though users find the generated descriptions highly beneficial for comprehension. AI
IMPACT This work provides a benchmark for evaluating and improving MLLMs' ability to generate accessible audio descriptions, potentially enhancing media accessibility for visually impaired users.
RANK_REASON The cluster contains a research paper detailing a new dataset and evaluation framework for multimodal large language models. [lever_c_demoted from research: ic=1 ai=1.0]
- arXiv
- Hugging Face
- Mind The Gap (MTG-AD)
- MLLMs
- MTG-AD-Eval
- Multimodal Large Language Models
- Subhashini Venugopalan
- user-generated content
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
