New framework evaluates NLP explanation robustness in black-box enterprise systems

A new framework for evaluating the robustness of explanations in enterprise NLP systems has been proposed. This framework uses a leave-one-out occlusion method to assess how stable token-level explanations are under various perturbations. The study found that larger decoder-based LLMs, such as Llama 70B, provide significantly more stable explanations than smaller encoder-based models, with improved stability correlating with model scale. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Provides a method for selecting more reliable NLP models for enterprise use, especially in compliance-sensitive applications.

RANK_REASON Academic paper proposing a new evaluation framework for NLP explanations.

Read on arXiv cs.CL →

• arXiv
• Guilin Zhang
• BERT
• RoBERTa
• Qwen 7B
• Qwen 14B
• Llama 8B
• Llama 70B

• paper
• safety