New benchmark HarmThoughts targets fine-grained harm detection in LLM reasoning

Researchers have introduced HarmThoughts, a new benchmark designed to detect harmful behavior at a granular, sentence-level within the reasoning traces of large language models. Existing safety evaluations often overlook how harm emerges during the multi-step reasoning process, focusing only on final outputs. HarmThoughts addresses this by using a taxonomy of 16 behavioral categories to annotate over 56,000 sentences from more than 1,000 reasoning traces. The benchmark aims to improve safety monitoring, intervention, and failure diagnosis by analyzing how reasoning steps contribute to or mitigate harmful outcomes, and evaluating the effectiveness of different monitoring approaches. AI

IMPACT Enables more precise identification and mitigation of emergent harmful behaviors in LLM reasoning processes.

RANK_REASON The cluster contains an academic paper introducing a new benchmark for AI safety research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• HarmThoughts
• Ishita Kakkar

• safety
• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →