MTEB
PulseAugur coverage of MTEB — every cluster mentioning MTEB across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
AI agents fail due to flawed search index distribution, not prompting
A common issue in AI agents is that their search results appear correct but lead to factually wrong answers due to problems with the underlying search index. This is not a prompting issue but a distribution problem, whe…
-
HAKARI-Bench offers lightweight evaluation for retrieval models · 2 sources tracked
Researchers have introduced HAKARI-Bench, a lightweight benchmark designed to streamline the evaluation of retrieval architectures and efficiency settings for retrieval-augmented generation and semantic search. This new…
-
New Slovak Text Embedding Benchmark and Models Released
Researchers have introduced SkMTEB, a new benchmark designed to evaluate text embedding models specifically for the Slovak language. This benchmark includes 31 datasets across 7 task types, significantly expanding cover…
-
New method enhances LLM text embeddings using text reversal
Researchers have introduced ReverseEOL, a novel method to enhance text embeddings generated by decoder-only Large Language Models (LLMs) without additional training. This technique augments standard embeddings by incorp…
-
New research explores extreme text embedding compression
Researchers have investigated the combined impact of dimensionality reduction and quantization on compressing text embeddings. Their experiments, using four MTEB task families and four pretrained embedding models, show …
-
PromptEmbedder offers efficient, transferable text embeddings via dual-LLM prompting
Researchers have introduced PromptEmbedder, a new dual-LLM framework designed to improve the efficiency and transferability of text embeddings. This method decouples embedding knowledge from specific model weights by us…
-
New benchmarks and studies probe multilingual text embedding robustness
Researchers are exploring the robustness of multilingual text embeddings across various tasks and languages. One study introduces new indicators to assess how dataset composition and ranking methods affect model perform…
-
Google DeepMind unveils Gemini Embedding 2 multimodal model
Google DeepMind has introduced Gemini Embedding 2, a new native multimodal embedding model. This model can generate unified representations for video, audio, image, and text data, demonstrating strong zero-shot capabili…
-
Embedding models' structure predicts benchmark performance, study finds
Researchers have demonstrated that the organization of embedding spaces within high-performing models consistently predicts their benchmark performance. By evaluating 25 embedding models across five MTEB tasks, they fou…
-
Hugging Face releases Ettin Reranker models for improved search
Hugging Face has released a new family of six Ettin Reranker models, built on top of Ettin ModernBERT encoders. These models offer state-of-the-art performance for their respective sizes and are designed for the retriev…
-
LMEB benchmark evaluates long-horizon memory retrieval beyond traditional passage retrieval
Researchers have introduced the Long-horizon Memory Embedding Benchmark (LMEB), a new evaluation framework designed to assess the capabilities of embedding models in handling complex, long-horizon memory retrieval tasks…
-
Causal2Vec enhances decoder-only LLMs for embeddings without architecture changes
Researchers have introduced Causal2Vec, a novel method to enhance decoder-only large language models (LLMs) for embedding tasks without altering their core architecture. This approach involves pre-encoding input text in…
-
EPIC training method boosts LLM text encoder performance on MTEB benchmark
Researchers have developed a new training strategy called EPIC (Embedding-based In-Context Prompt Training) to improve the quality of text embeddings generated by large language models. This method reduces computational…
-
Hugging Face launches MTEB benchmark for Polish text embeddings
Researchers have introduced the Polish Massive Text Embedding Benchmark (PL-MTEB), a new evaluation suite designed to assess text embedding models specifically for the Polish language. This benchmark includes 30 diverse…
-
OpenAI launches new embedding models with price cuts and performance boosts
OpenAI has released new embedding models, text-embedding-3-small and text-embedding-3-large, offering significant improvements in performance and efficiency over previous models like text-embedding-ada-002. These new mo…