A new empirical study investigates the challenge of catastrophic forgetting in coding agents that undergo continual harness self-evolution. Researchers found that as agents evolve to handle new tasks, their performance on previously learned tasks can significantly degrade. For instance, an agent using DeepSeek-V4-Flash-0731 and Meta-Harness showed a sharp decline in SWE-Bench performance after evolving on FeatureBench. The findings highlight the need for methods to enable sustained self-evolution without performance loss on earlier tasks. AI
IMPACT Highlights a critical limitation in current AI agent evolution, suggesting a need for new techniques to prevent performance degradation on previously learned tasks.
RANK_REASON The cluster contains an academic paper detailing empirical findings on AI model behavior. [lever_c_demoted from research: ic=1 ai=1.0]
Read on Hugging Face Daily Papers →
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
