Vector RAG vs LLM-Compiled Wiki: A Preregistered Comparison on a Small Multi-Domain Research
A new research paper compares Vector Retrieval-Augmented Generation (RAG) against an LLM-compiled wiki for answering questions over a small corpus of 24 research papers. While the wiki excelled at synthesizing information across multiple documents, RAG performed better on single-fact lookups and overall groundedness. Exploratory analyses revealed the wiki offered stronger claim-level citation support, but a modified RAG approach could match the wiki's cross-paper synthesis capabilities at a lower cost. The study concludes that effective research synthesis involves distinct capabilities like evidence organization, citation accuracy, and cost-efficiency, with no single architecture excelling in all areas. AI
IMPACT Compares RAG and LLM-compiled wikis for research synthesis, highlighting trade-offs in cost, accuracy, and synthesis capabilities.