A recent A/B test evaluated nine popular AI agent skills, finding that four of them provided no measurable benefit. The study focused on how skills, which are essentially `.md` files that teach coding agents like Claude Code or Gemini CLI new capabilities, perform with current LLM models. Results indicated that skills which introduce new workflows were effective, while those merely restating good coding practices offered no improvement. Smaller models appeared to benefit more from these skills than larger ones, and the description of a skill alone could sometimes influence model behavior. AI
IMPACT This analysis highlights that not all AI agent skills are effective, suggesting developers should carefully vet them for actual utility, especially for smaller models.
RANK_REASON The item discusses the effectiveness of third-party tools (agent skills) for AI models, rather than a core AI release or research.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
