PulseAugur
EN
LIVE 22:47:45

RaguTeam wins SemEval-2026 LLM task with judge-orchestrated ensemble

By PulseAugur Editorial · [2 sources] ·

RaguTeam has developed a winning system for the SemEval-2026 Task 8, which focuses on faithful multi-turn response generation. Their approach utilizes a heterogeneous ensemble of seven large language models, with a GPT-4o-mini acting as a judge to select the best response. This ensemble method outperformed 26 other teams, achieving a harmonic mean of 0.7827 and demonstrating the effectiveness of diverse model families and prompting strategies. AI

IMPACT Demonstrates an effective ensemble strategy for multi-turn response generation, potentially influencing future research in faithful dialogue systems.

RANK_REASON This is a research paper detailing a system's performance in a specific academic task.

Read on arXiv cs.LG →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

RaguTeam wins SemEval-2026 LLM task with judge-orchestrated ensemble

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
This is a research paper detailing a system's performance in a specific academic task.
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
143 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [2]

  1. arXiv cs.LG TIER_1 English(EN) · Ivan Bondarenko, Roman Derunets, Oleg Sedukhin, Mikhail Komarov, Ivan Chernov, Mikhail Kulakov ·

    RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generation

    arXiv:2605.04523v1 Announce Type: cross Abstract: We present our winning system for Task~B (generation with reference passages) in SemEval-2026 Task~8: MTRAGEval. Our method is a heterogeneous ensemble of seven LLMs with two prompting variants, where a GPT-4o-mini judge selects t…

  2. arXiv cs.CL TIER_1 English(EN) · Mikhail Kulakov ·

    RaguTeam at SemEval-2026 Task 8: Meno and Friends in a Judge-Orchestrated LLM Ensemble for Faithful Multi-Turn Response Generation

    We present our winning system for Task~B (generation with reference passages) in SemEval-2026 Task~8: MTRAGEval. Our method is a heterogeneous ensemble of seven LLMs with two prompting variants, where a GPT-4o-mini judge selects the best candidate per instance. We ranked 1st out …

RELATED ENTITIES

RELATED TOPICS