SafeCap framework enhances LVLM safety via caption-mediated reinforcement learning

Researchers have developed SafeCap, a new reinforcement learning framework designed to enhance the safety of Large Vision-Language Models (LVLMs). This method trains a policy model to generate safety-relevant image captions before producing a final answer. The caption generation is optimized by how well it guides a frozen LLM towards a safe decision, encouraging the model to expose visual cues critical for safe responses. Evaluations on multiple safety and utility benchmarks show SafeCap significantly improves safety performance across various model settings while maintaining or enhancing vision utility, outperforming existing methods like SFT, DPO, and SafeGRPO. AI

IMPACT This research could lead to more robust safety mechanisms in multimodal AI systems, reducing vulnerabilities to adversarial attacks.

RANK_REASON The cluster contains a research paper detailing a new method for improving AI model safety. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• arXiv
• DirectCap
• Direct Preference Optimization
• LVLMs
• SafeCap
• SafeGRPO
• supervised fine-tuning

• paper
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →