New framework guides LLMs to choose between RAG and long-context processing

Researchers have developed a new framework called Pre-Route to help large language models decide whether to use retrieval-augmented generation (RAG) or long-context (LC) processing for document understanding. This proactive system uses lightweight metadata to analyze tasks, estimate coverage, and predict information needs, leading to more explainable and cost-effective routing decisions. Experiments show that Pre-Route outperforms existing methods on benchmarks like LaRA and LongBench-v2, demonstrating that LLMs have latent routing abilities that can be effectively elicited and even distilled into smaller models. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

IMPACT Improves efficiency and explainability in LLM document processing, potentially reducing costs for long-context tasks.

RANK_REASON The cluster contains an academic paper detailing a new framework and experimental results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• Pre-Route
• LLMs
• RAG
• LC
• LaRA
• LongBench-v2

• paper
• model release