AI coding agent commits detectable without signatures, but imperfectly

A new arXiv paper explores the challenge of distinguishing between code commits made by AI agents and those made by human developers when authorship signatures are absent. Researchers found that while models like GPT-6 Sol can achieve an AUC of up to 0.859 in detecting AI commits based on factors like message verbosity and change size, this detection is not perfect, with a precision of only 0.332 at the observed AI commit share. The study also highlighted that detection performance varies significantly across different AI agents, with poor transferability between models like Claude Code and GitHub Copilot, indicating that signature-free detection can aid in prioritizing commits for review but cannot fully replace traditional attribution methods. AI

IMPACT Highlights the growing challenge of attributing code authorship as AI agents become more integrated into software development workflows.

RANK_REASON Research paper on AI model capabilities. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• alphaXiv
• arXiv
• CatalyzeX
• Claude Code
• Cursor
• DagsHub
• GitHub Copilot
• Gotit.pub
• GPT-6 Sol
• Hugging Face
• ScienceCast
• tf–idf
• XGBoost

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →