The Cheaper API Was 2.5x Cheaper. It Cost 1.6x More.
A developer discovered that the seemingly cheaper API tier for an LLM was actually more expensive per successful task due to a higher failure rate. The cost per successful task is calculated by dividing the cost per attempt by the success rate. This means that an API with a lower per-call price but a higher failure rate can end up costing significantly more when accounting for the necessary retries. AI
IMPACT Highlights the importance of considering API success rates, not just per-call prices, for cost-effective LLM agent development.