LLM input tokens account for 99.2% of costs, analysis finds

A recent analysis of over 28,000 LLM calls reveals that the vast majority of costs, approximately 99.2%, are associated with input tokens rather than model output. This finding suggests that optimizing the model itself, prompts, or temperature settings has a minimal impact on overall expenses. The true cost driver is the number of times a task is re-processed or forwarded, with a single task potentially consuming around 18 million tokens, of which only a small fraction is generated by the model. The report emphasizes that the cost is determined by the specific content of the tool schemas used, not simply the number of tools available, challenging common market intuitions about cost reduction. AI

IMPACT Highlights that input token management and task re-processing are critical cost factors for LLM applications, rather than model efficiency alone.

RANK_REASON Analysis of LLM token costs based on empirical data and reproducible commands. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• bb7190b0
• git cat-file blob bb7190b0
• REPORTE_TOKENS.md
• /tmp/panel.md
• /tmp/ttr.json
• tools/reporte_tokens.py

• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →