PulseAugur
EN
LIVE 05:55:49

AI coding agents' pull requests show mixed success, high rejection rates

By PulseAugur Editorial · [4 sources] ·

Two new research papers analyze the effectiveness of AI agents in software development, specifically focusing on pull requests. The first paper, "Toward Instructions-as-Code," found that while instruction files can guide AI agents like GitHub Copilot, their impact on pull request success rates is mixed, with some projects seeing improvements and others experiencing declines. The second paper, "Understanding the Rejection of Fixes Generated by Agentic Pull Requests," investigated why AI-generated fixes are rejected, identifying incorrect implementations, CI pipeline failures, and agent limitations as key reasons, with nearly half of all proposed fixes being discarded. AI

IMPACT Research highlights challenges in AI agent integration, suggesting a need for better guidance and task prioritization to improve efficiency and reduce wasted effort in software development.

RANK_REASON Two academic papers published on arXiv analyzing AI agent performance in software engineering.

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 4 sources. How we write summaries →

COVERAGE [4]

  1. arXiv cs.AI TIER_1 English(EN) · Ali Arabat, Mohammed Sayagh ·

    Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests

    arXiv:2606.13449v1 Announce Type: cross Abstract: AI-agents (e.g., GitHub Copilot) collaborate as teammates in different software engineering tasks, including code generation proposed through pull requests (Agentic-PRs). For better agent efficiency, developers create instruction …

  2. arXiv cs.AI TIER_1 English(EN) · Mahmoud Abujadallah, Ali Arabat, Mohammed Sayagh ·

    Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset

    arXiv:2606.13468v1 Announce Type: cross Abstract: AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects. From a first exploration of the AIDev dataset, we find that 46.41\% of the fixes proposed by the agents Copilot, …

  3. arXiv cs.AI TIER_1 English(EN) · Mohammed Sayagh ·

    Understanding the Rejection of Fixes Generated by Agentic Pull Requests -- Insights from the AIDev Dataset

    AI coding agents are increasingly used to generate pull requests (PRs) that propose code fixes in software projects. From a first exploration of the AIDev dataset, we find that 46.41\% of the fixes proposed by the agents Copilot, Devin, Cursor, and Claude are rejected. This repre…

  4. arXiv cs.AI TIER_1 English(EN) · Mohammed Sayagh ·

    Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests

    AI-agents (e.g., GitHub Copilot) collaborate as teammates in different software engineering tasks, including code generation proposed through pull requests (Agentic-PRs). For better agent efficiency, developers create instruction files that guide the AI-agents, including how to n…

RELATED ENTITIES

RELATED TOPICS