Researchers have investigated the impact of different training and calibration data on the performance of three open-source AI models on the ARC-AGI-2 reasoning benchmark. For the NVARC Qwen3-4B model, a corpus of 12,000 executable rules and 95,993 tasks was developed, with fine-tuning on real task records improving its score from 28.96 to 33.06. The Qwen3.8-27B reasoning model saw its score increase from 26.81 to 44.03 through an enhanced inference procedure, which included a 'thinking cap' and program checks. The Qwen3.8-Flash-Next mixture-of-experts model achieved the best result of 56.67 by optimizing its expert layers and utilizing an FP8 indexer cache with a program-checking tool. AI
IMPACT Demonstrates how data and inference strategies can significantly boost reasoning capabilities in open-source AI models.
RANK_REASON Research paper detailing training data and inference techniques for open models on a specific benchmark. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
