Open-weight AI models match closed models on security benchmark at lower cost

A new study published on arXiv evaluates twenty-six language models, both open-weight and closed, on the CTI-REALM benchmark for security work. The research found that open-weight models can achieve performance comparable to expensive closed models at a fraction of the cost, especially when given more turns to complete tasks. The study also suggests that routing different stages of the workflow to specialized models could be more effective than relying on a single model's overall score. AI

IMPACT Suggests open-weight models can be cost-effective for specialized tasks, potentially shifting enterprise adoption strategies.

RANK_REASON The cluster contains an academic paper detailing research findings on AI model performance. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv:2603.13517
• CatalyzeX
• CORE Recommender
• CTI-REALM
• DagsHub
• Gotit.pub
• Hugging Face
• ScienceCast

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →