Collinear AI has released version 2 of its CWE-bench, a benchmark designed to test the cybersecurity capabilities of AI models. This new version includes 100 tasks, covering 91 different Common Weakness Enumerations (CWEs), and is now publicly available for anyone to run and evaluate their models. The benchmark uses a programmatic scoring system for reproducibility and has been refined with LLM judge panels for fairness and difficulty. In the latest results, Gemini 4 Argon leads with a 67.8% pass@1 rate, closely followed by Claude Opus 5.5, indicating that while progress is being made, the long tail of cybersecurity vulnerabilities remains largely unsolved by current AI models. AI
IMPACT Establishes a new benchmark for AI cybersecurity capabilities, highlighting current model limitations and driving further research.
RANK_REASON Release of a new version of a benchmark for evaluating AI capabilities. [lever_c_demoted from research: ic=1 ai=1.0]
- Claude Opus
- Claude Opus 5.5
- Collinear AI
- Common Weakness Enumeration
- CWE-bench
- DeepSeek V4.1 Flash
- Gemini
- Gemini 4 Argon
- generative pre-trained transformer
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
