LMEB benchmark evaluates long-horizon memory retrieval beyond traditional passage retrieval

Researchers have introduced the Long-horizon Memory Embedding Benchmark (LMEB), a new evaluation framework designed to assess the capabilities of embedding models in handling complex, long-horizon memory retrieval tasks. Unlike existing benchmarks that focus on traditional passage retrieval, LMEB incorporates 22 datasets and 193 zero-shot tasks across four distinct memory types: episodic, dialogue, semantic, and procedural. Initial evaluations of 15 models indicate that LMEB presents a suitable challenge, that larger model size does not guarantee better performance, and that LMEB measures different capabilities than the MTEB benchmark. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Introduces a new benchmark that may drive development of models better suited for long-term, context-dependent memory retrieval.

RANK_REASON The cluster describes a new academic paper introducing a benchmark for evaluating AI models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• LMEB
• OpenClaw
• MTEB
• Xinping Zhao

• paper
• other