MCP retrieval tools increase token costs on small repos, save tokens on large ones

A replication of the CodeNib paper's agent experiment revealed that using MCP retrieval tools can significantly increase token costs compared to traditional grep commands, especially on smaller codebases. While the CodeNib paper claimed substantial token savings, this replication found that on a 33-file repository, the MCP tools used 4.1 times more tokens. However, the situation reversed on a larger 249-file repository, where the MCP tools resulted in an 86% token saving, indicating that repository size is a critical factor in the efficiency of these retrieval methods. AI

IMPACT Highlights the critical role of repository size in determining the efficiency of AI coding agent tools.

RANK_REASON Replication of a research paper's experiment with differing results. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — MCP tag →

• arXiv:2607.25431
• Claude Haiku 4.5
• CodeNib
• Gemma 4-12B
• Grep Command
• MCP

• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →