EngramaBench evaluates long-term conversational memory for LLMs

Researchers have introduced EngramaBench, a new benchmark designed to evaluate the long-term conversational memory capabilities of large language models. The benchmark features five distinct personas and one hundred multi-session conversations, with queries testing factual recall, temporal reasoning, and synthesis. In evaluations, GPT-4o with full-context prompting achieved the highest overall score, though a graph-structured memory system called Engrama demonstrated superior performance in cross-space reasoning. AI

IMPACT Introduces a new benchmark for evaluating LLM long-term memory, potentially guiding future memory system development.

RANK_REASON This is a research paper introducing a new benchmark for evaluating LLM memory.

Read on arXiv cs.CL →

• Engrama
• EngramaBench
• GPT-4o

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →