New SCOUT system slashes LLM tool token usage by 99%

A new paper introduces SCOUT, a system designed to optimize tool discovery and context management for large language model (LLM) agents. SCOUT addresses the challenge of LLM context windows becoming saturated with tool schemas by selectively injecting only relevant tools for the current task. It utilizes a hybrid retrieval method combining BM25 and dense vector search to identify the most pertinent tools from a large catalog. Implemented at PayPal, SCOUT has demonstrated a significant reduction in tool-token consumption, decreasing it by 99% and substantially cutting per-query inference costs at an enterprise scale. AI

影响 This system significantly reduces LLM inference costs and improves efficiency for enterprise applications by optimizing tool selection.

排序理由 The cluster contains a research paper detailing a new system and its implementation. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.AI 阅读 →

• BM25
• Large language model (LLM) agents
• Model Context Protocol (MCP)
• PayPal
• Reciprocal Rank Fusion
• SCOUT

• 论文
• 基础设施

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →