Brief

A new technique called RAG Rerank significantly improves the accuracy of retrieval-augmented generation (RAG) systems by adding a reranking step. Standard RAG retrieves documents based on vector similarity, which can lead to irrelevant documents being prioritized. RAG Rerank uses a cross-encoder model to re-evaluate the relevance of a shortlisted set of documents, ensuring that the most pertinent information is passed to the language model. This approach enhances accuracy at the cost of slightly increased latency and expense, making it particularly valuable for applications where incorrect answers are costly. AI

Hugging Face has released a new family of six Ettin Reranker models, built on top of Ettin ModernBERT encoders. These models offer state-of-the-art performance for their respective sizes and are designed for the retrieve-then-rerank pattern in information retrieval systems. The release includes the models, their training data, and a full training recipe, enabling users to integrate them or even train their own rerankers. AI

Setting up a local LLM knowledge base often yields poor results due to issues in the retrieval pipeline, not the model itself. Common problems include inadequate chunking that splits sentences or groups unrelated content, using an embedding model that doesn't capture semantic nuances for specific domains, and retrieving too few chunks to reconstruct the necessary context. Solutions involve using recursive splitters with overlap and semantic boundaries for better chunking, testing various embedding models like BAAI/bge-base-en-v1.5 or intfloat/e5-base-v2 to find one suited to the data, and increasing the number of retrieved chunks or employing reranking to ensure comprehensive context. AI