Simon Willison has developed "smevals," a new evaluation suite designed to assess the capabilities of AI models, prompts, and harnesses. This tool, built in collaboration with Jesse Vincent's Prime Radiant applied AI research lab, allows users to create and run evaluation suites against various models, grade the results, and generate reports. Willison describes smevals as his third iteration on an evals framework, aiming to provide a more effective approach for evaluating AI performance. AI
IMPACT Provides a new framework for developers to test and compare AI model performance.
RANK_REASON The item describes a new tool for evaluating AI models, not a frontier model release or significant industry event.
Read on Mastodon — mastodon.social →
- Claude Code
- Claude Opus-4.6
- GPT-5.5
- Hugging Face
- Jesse Vincent
- Kimi k3
- OpenAI
- Prime Radiant
- Simon Willison
- smevals
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
