Large language models, including GPT-4 and Llama 2, exhibit a phenomenon known as "Agent Compliance Decay," where safety rules embedded in system prompts are ignored once they fall outside the model's context window. This decay is quantifiable and occurs across various LLM APIs, with compliance dropping significantly as the dialogue length increases. For instance, GPT-4 maintained compliance up to approximately 6,000 tokens but saw a sharp decline thereafter, while Llama 2 and MPT models showed earlier signs of rule violation. This issue poses a risk for LLM agents relying on persistent safety instructions, as standard memory management techniques may not preserve these critical rules. AI
IMPACT This research highlights a critical limitation in current LLM agent design, necessitating new approaches to ensure consistent safety and compliance in long-running applications.
RANK_REASON The item details research into a specific LLM behavior and provides experimental data. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
