A recent analysis of LLM agent performance on the THOL leaderboard suggests that the number of tools available to an agent does not significantly impact the cost per solved task. While a previous estimate suggested a high cost for agents using many tools, real-world data shows that even agents with up to 30 tools incur costs similar to those with no tools. The primary driver of cost appears to be the complexity of the task and the number of turns required, rather than the sheer quantity of tools. AI
IMPACT Suggests that optimizing LLM agent cost should focus on task complexity and turns, not just the number of tools.
RANK_REASON Analysis of existing data and commentary on prior findings.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →