PulseAugur
EN
LIVE 13:57:19

LLMs evaluated on Scrum certification questions; Gemini 3 Flash leads accuracy

By PulseAugur Editorial · [2 sources] ·

A recent study evaluated the performance of three large language models—GPT-5 mini, Gemini 3 Flash, and DeepSeek Chat 3.2—on 993 Scrum certification-style questions. Gemini 3 Flash demonstrated the highest accuracy, while all models showed low intra-model variability. Performance varied by question format and topic, with models excelling in normatively explicit areas and single-answer multiple-choice questions, but struggling with multi-select and True/False formats, as well as more interpretive Scrum topics. The analysis revealed systematic errors, including overgeneralization and conflicts between common interpretations and strict Scrum definitions. AI

IMPACT LLM performance on domain-specific certification questions varies, highlighting the need for careful prompting and evaluation for reliable use in professional training.

RANK_REASON The cluster consists of two academic papers presenting empirical research on LLM performance on a specific domain.

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

LLMs evaluated on Scrum certification questions; Gemini 3 Flash leads accuracy

COVERAGE [2]

  1. arXiv cs.AI TIER_1 English(EN) · Robson Alves Vilar, Emanuel Dantas Filho, Ademar Fran\c{c}a de Sousa Neto, Mirko Perkusich, Danyllo Wagner Albuquerque, Jo\~ao Paiva, Kyller Gorg\^onio, Angelo Perkusich ·

    Comparing Large Language Models on Scrum Certification-Style Questions: Accuracy, Stability, and Error Patterns

    arXiv:2607.00048v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in exam- and certification-style question answering tasks, where their ability to retrieve, interpret, and apply domain-specific knowledge can be systematically assessed. In Softw…

  2. arXiv cs.AI TIER_1 English(EN) · Mirko Perkusich, Danyllo Albuquerque, Jo\~ao Paiva, Robson Vilar, Emanuel Dantas, Ademar Fran\c{c}a de Sousa Neto, Rohit Gheyi, Kyller Gorg\^onio, Angelo Perkusich ·

    Prompting GPT-5 on Scrum Certification Questions: An Empirical Accuracy Study

    arXiv:2607.00049v1 Announce Type: cross Abstract: Large Language Models (LLMs) are increasingly used in Agile Software Development for documentation, coaching, and training. As practitioners adopt these tools to prepare for certifications such as Professional Scrum Master (PSM), …

RELATED ENTITIES

RELATED TOPICS