A new study published on arXiv investigates hallucinations in Large Language Models (LLMs) when generating UML class diagrams from natural language requirements. Researchers developed a taxonomy to categorize these hallucinations into Prompt Faithfulness, Code Faithfulness, and Reasoning Quality, finding that Code Faithfulness was more common in DeepSeek-R1 while Prompt Faithfulness dominated in DeepSeek-V4-Pro. The study also found that longer reasoning traces did not correlate with more hallucinations, but higher requirement complexity was negatively associated with Prompt Faithfulness errors. Mitigation experiments showed that adding constraints to the generation prompt improved F1 scores, while post-hoc code review offered no consistent benefit. AI
IMPACT Identifies specific failure modes in LLMs for code generation, potentially leading to more reliable diagramming tools.
RANK_REASON Academic paper detailing a new taxonomy and mitigation study for LLM hallucinations. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
