SPI: Query-Depth-Adaptive Indexing for Streaming RAG in Vector Databases
Researchers have introduced Semantic Pyramid Indexing (SPI), a novel indexing framework for vector databases designed to enhance retrieval-augmented generation (RAG) pipelines. SPI adapts the retrieval depth based on query complexity and semantic granularity, organizing embeddings into multiple resolution levels. This approach allows for efficient streaming insertion of new vectors without full index rebuilds and supports progressive coarse-to-fine searches. AI