Researchers have introduced the Precise Debugging Benchmark (PDB) framework to evaluate the debugging capabilities of large language models. The framework converts existing coding datasets into debugging benchmarks, automatically generating buggy programs with synthesized atomic bugs. PDB employs novel metrics like edit-level precision and bug-level recall to assess how accurately models fix code. Experiments revealed that leading models like GPT-5.1-Codex and DeepSeek-V3.2-Thinking, despite high test pass rates, struggle with precision, often over-editing solutions. AI
IMPACT Highlights a gap in current LLM coding abilities, suggesting a need for new post-training methods to improve precise debugging.
RANK_REASON New academic paper introducing a benchmark and evaluation metrics for LLM debugging capabilities.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
