A new study published on arXiv compares the effectiveness of fine-tuned open large language models (LLMs) against traditional methods like TF-IDF and CodeBERT for classifying bug severity in Eclipse reports. The research found that while fine-tuned LLMs like Llama 3 8B and Qwen 2.5 7B achieved higher macro-F1 scores than zero-shot LLMs and CodeBERT, their performance was comparable to or slightly below TF-IDF models in some configurations. The study also highlighted significant differences in how these models handle specific bug severity classes, with fine-tuned Llama 3 excelling at identifying critical issues but struggling with minor ones, while Qwen 2.5 showed the opposite pattern. AI
IMPACT Fine-tuned open LLMs show potential for specialized tasks like bug severity classification, though they may not consistently outperform traditional methods without further optimization.
RANK_REASON Academic paper comparing LLM performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
- Bugzilla
- CodeBERT
- CyberTriage
- Eclipse Foundation
- Hugging Face
- Llama 3 8B
- open LLMs
- QLoRA
- Qwen 2.5 7B
- TF-IDF
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
