A new empirical study investigates how Large Language Models (LLMs) process bug reports for automated program repair. Researchers analyzed attention patterns in LLMs across 319 Python and Java bugs from SWE-bench Verified and Multi-SWE-bench. The study found that successful repairs often involve diffused attention across various bug report components like descriptions and stack traces, whereas failures tend to show over-localized attention on metadata such as version information. This research provides the first evidence that misallocated attention is a significant factor in LLM-based automated program repair failures, offering insights for developing more reliable systems. AI
IMPACT Provides insights into improving LLM reliability for code-related tasks like bug fixing.
RANK_REASON Academic paper detailing an empirical study of LLM behavior on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
