RAG pipeline costs: Embedding is cheap, vector DBs and LLMs are expensive

A developer investigated the cost breakdown of retrieval-augmented generation (RAG) pipelines, challenging the common assumption that embedding is the most expensive component. Through detailed analysis, it was found that embedding documents, even at scale, is relatively inexpensive. The primary cost drivers are identified as the vector database infrastructure, which runs continuously, and the final stage where a large language model processes retrieved chunks to generate an answer. AI

IMPACT Highlights that vector database infrastructure and LLM processing are the main cost centers in RAG, not embedding.

RANK_REASON Developer's personal analysis of RAG costs, not a primary source release or industry-wide event.

Read on dev.to — LLM tag →

• OpenAI
• Redis
• retrieval-augmented generation

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →