An investigation into a benchmark for AI agent citation survival revealed that the OpenAI Agents SDK, when using GPT-4o mini, appeared to lose nearly half its citations compared to LangGraph. This discrepancy was not due to the agent loop, as both frameworks had similar loop counts and LangGraph maintained its citations. The primary suspect for the citation loss in the OpenAI Agents SDK was identified as the default temperature setting, which allowed for more randomness in the model's output compared to LangGraph's temperature of 0. AI
IMPACT Investigates potential issues with citation handling in AI agent frameworks, impacting reliability of RAG systems.
RANK_REASON Analysis of an AI agent framework's citation performance. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →