A study analyzing developer feedback on CodeRabbit's agentic code reviews found that a significant portion of these AI-generated comments are rejected. Out of 31,073 reviewed pull requests, 56.3% of CodeRabbit's suggestions were rejected, primarily due to inaccuracies, irrelevance, or misalignment with developer intent. While agentic reviews focused more on functional concerns, they were less effective than human reviews in addressing evolvability. The research suggests opportunities for improvement in agentic code review tools, with machine learning models showing potential in predicting review rejection. AI
IMPACT Highlights current limitations in agentic code review tools, suggesting areas for improvement in accuracy and relevance for AI-assisted development.
RANK_REASON Academic paper presenting empirical study of an AI tool's effectiveness. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
