New pipeline scores educational data for LLM pre-training

Researchers have developed Edu-QuRating, a new pipeline for multi-dimensional educational data scoring and curation. This system defines education-specific rubrics and uses an LLM judge to label document pairs, distilling these preferences into reusable Edu-QuRaters. These raters can score text chunks on various educational criteria, achieving high accuracy in predicting pairwise judgements. The pipeline has been applied to filter pre-training corpora for small language models, resulting in improved aggregate accuracy on benchmarks, and to enhance GRPO post-training by using Edu-QuRater scores as reward terms, leading to preferred responses in pedagogical quality and instruction following. AI

IMPACT Enhances LLM pre-training by enabling more nuanced educational data filtering and reward shaping.

RANK_REASON The cluster describes a new research paper detailing a novel method for data curation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Edu-QuRaters
• Edu-QuRating
• FineWeb-Edu
• FineWeb-Edu-Fortified
• GPT-4.1 mini
• GRPO
• QuRating
• Qwen3-4B
• Wettig et al.

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →