LLMs hallucinate software packages, posing supply chain risks; defenses evaluated

Researchers are investigating the problem of Large Language Models (LLMs) hallucinating non-existent software packages when generating code. This issue poses a risk to the software supply chain by creating exploitable vulnerabilities. Studies have evaluated various defense mechanisms, including guided decoding strategies, self-refinement, and Retrieval-Augmented Generation (RAG), to mitigate these hallucinations. Findings indicate that while RAG can reduce hallucination rates, its effectiveness varies across different models and languages, and it must be balanced against the utility of the generated code. Furthermore, adversarial prompts significantly increase hallucination rates, highlighting the need for robust defenses that can handle actively hostile inputs. AI

IMPACT Highlights critical vulnerabilities in LLM code generation and evaluates defenses, impacting the security of AI-assisted software development.

RANK_REASON The cluster contains two academic papers detailing research into LLM code generation vulnerabilities and defense mechanisms.

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• LibHalluBench
• Library Hallucinations
• LLMs
• Lukas Twist
• ScienceCast
• Greedy decoding
• Iyiola E. Olatunji
• JavaScript
• Package Hallucination
• Python
• Retrieval-Augmented Generation (RAG)
• Ruby
• Rust
• Self-Refine
• software supply chain

• paper
• safety
• product

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →