SemiAnalysis analyzed over 1.5 million requests made to Anthropic's Claude models, primarily Opus. Their findings indicate that approximately 95% of tokens used were cache hits, resulting in an estimated 84% reduction in their token costs. The analysis also noted that Claude Haiku likely has a lower cache hit rate due to its use in shorter-lived sessions or as a sub-agent. AI
IMPACT Highlights the significant cost-saving potential of caching mechanisms in large language models for developers.
RANK_REASON Analysis of usage data from a third-party regarding model performance and cost.
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
