New benchmark reveals vision-language models struggle with nuanced driving hazards

A new benchmark, BD-HazardVLM, has been developed to evaluate the defensive-driving reasoning capabilities of vision-language models (VLMs) in the context of Bangladesh road scenes. The benchmark, comprising 500 images from the RSUD20K and TFP-BD datasets, includes annotations for hazard presence, category, risk timing, severity, defensive actions, and potential biases. When tested on this benchmark, models like Qwen2.5-VL-7B, InternVL3-8B, and MiniCPM-V-4.6 demonstrated strong performance in basic hazard recognition but struggled significantly with fine-grained reasoning, such as accurately categorizing hazards or predicting risk timing. AI

IMPACT Highlights limitations in current vision-language models for safety-critical applications like autonomous driving, particularly in diverse global contexts.

RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Abdullah Al Maruf
• Bangladesh
• BD-HazardVLM
• InternVL3-8B
• MiniCPM-V-4.6
• Qwen2.5-VL-7B
• RSUD20K
• TFP-BD

• paper
• safety
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →