GraphRAG on Consumer Hardware: Benchmarking Local LLMs for Healthcare EHR Schema Retrieval
Researchers evaluated the GraphRAG pipeline for retrieving information from Electronic Health Record (EHR) schemas using open-source large language models deployed on consumer hardware. The study benchmarked models like Llama 3.1, Mistral, Qwen 2.5, and Phi-4-mini on a single GPU, assessing indexing efficiency, knowledge graph construction, latency, and answer quality. Results indicated that models below approximately 7 billion parameters struggle with structured output errors, and local retrieval generally outperformed global summarization in terms of speed and factual accuracy. AI
IMPACT Demonstrates the feasibility of using smaller, locally deployed LLMs for complex tasks like EHR schema retrieval, potentially improving privacy and reducing costs in healthcare.