We Had LLMs Hallucinating Legal URLs in Production — Here's What We Tried
A developer encountered a significant issue where LLMs hallucinated legal URLs in a RAG application, citing non-existent sections or subtly mutating real URLs. Standard prompt engineering and context reduction techniques proved insufficient to resolve the problem. The developer found success by replacing real URLs with short, opaque numeric codes before sending them to the LLM, then mapping these codes back to the original URLs after the LLM generated its response. AI
IMPACT Developers can use code substitution to prevent LLMs from hallucinating URLs, improving the reliability of RAG systems.