PulseAugur
EN
LIVE 00:11:14

Developer ditches semantic embeddings for BM25 in AI agent tool selection

By PulseAugur Editorial · [1 sources] ·

A developer building AI agents found that semantic embeddings, commonly used for tool selection, were unreliable in production. These embeddings struggled to differentiate between tools with similar descriptions, leading to incorrect tool choices. After testing three retrieval strategies, BM25-based search proved most effective, achieving 81% top-1 accuracy by indexing tool names, descriptions, and schema fields. AI

IMPACT Highlights the limitations of standard semantic search for structured tool selection in AI agents, advocating for keyword-based methods like BM25.

RANK_REASON Developer shares findings from testing retrieval strategies for AI agent tool selection, comparing semantic embeddings and BM25. [lever_c_demoted from research: ic=1 ai=0.7]

Read on r/MachineLearning →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

Developer ditches semantic embeddings for BM25 in AI agent tool selection

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Developer shares findings from testing retrieval strategies for AI agent tool selection, comparing semantic embeddings and BM25. [lever_c_demoted from research: ic=1 ai=0.7]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
product, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
110 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [1]

  1. r/MachineLearning TIER_1 English(EN) · /u/AbjectBug5885 ·

    Why I stopped using semantic embeddings for tool selection and switched back to BM25 [D]

    <!-- SC_OFF --><div class="md"><p>I've been building agents for about a year and recently shipped one for a client running ~140 MCP-exposed tools at peak. Along the way I made the canonical mistake. I used cosine similarity over tool description embeddings to pick which tools the…

RELATED ENTITIES

RELATED TOPICS