Kimi K3 lags US models in cyber exploits, possible distillation cited

Moonshot AI's Kimi K3 model demonstrated significantly weaker performance on offensive cyber tasks compared to leading U.S. models, scoring 32% on ExploitBench versus 76%. The model's safeguards also failed to prevent exploit development or simulated attacks. This disparity between Kimi K3's general benchmark scores and its cyber capabilities aligns with claims that Moonshot AI may have used distillation techniques, potentially involving Anthropic's models. AI

IMPACT Highlights potential vulnerabilities in AI models used for cyber security and suggests that distillation techniques may impact specialized capabilities.

RANK_REASON Research report on AI model performance in cyber security tasks. [lever_c_demoted from research: ic=1 ai=1.0]

Read on The Decoder →

• Anthropic
• British AI Security Institute
• Kimi K3
• Moonshot AI
• The Decoder
• U.S. Center for AI Standards and Innovation

• model release
• safety

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →