LLMs achieve high accuracy in classifying code commits via prompt engineering

Researchers explored using large language models (LLMs) for classifying conventional commits without requiring model fine-tuning. They evaluated zero-shot, few-shot, and chain-of-thought prompting strategies on Mistral-7B-Instruct, LLaMA-3-8B, and DeepSeek-R1-32B models. The study found that few-shot prompting yielded the highest accuracy, and the DeepSeek-R1-32B model performed best, indicating that larger models are more effective for this task. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Offers a training-free approach to commit classification, potentially reducing overhead for software maintenance and automation tools.

RANK_REASON Academic paper presenting novel methodology for commit classification using LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• Mistral-7B-Instruct
• LLaMA-3-8B
• DeepSeek-R1-32B
• InfluxDB

• paper
• other