Can Generalist Agents Automate Data Curation?
Researchers have developed Curation-Bench, a new benchmark designed to test whether generalist AI agents can automate the data curation process for AI development. In vision-language instruction tuning tasks, agents showed an ability to perform the curation loop but struggled with exploring new policy families, instead focusing on local variations. When provided with methodological guidance and adaptation scaffolds, agents were able to autonomously compose a data-selection policy that surpassed existing baselines with a significantly smaller data budget, highlighting the need for structured adaptation rather than simple prompting. AI
IMPACT Demonstrates a path toward automating a critical, labor-intensive aspect of AI development, potentially accelerating model training and improving efficiency.
