Brief

Free LLM endpoints, even those with consistent names, can degrade in reliability for tool-use tasks over time without notice. A weekly testing regimen is crucial for identifying these silent failures, as chat benchmark scores do not reflect a model's ability to consistently produce valid function calls. Models like Qwen3-next-80b and Qwen3-coder have shown zero success in recent tool-use tests, while Nemotron currently demonstrates high reliability. AI

A weekly test of free LLMs for tool-use reliability revealed significant decay in model performance over time. Two models, Qwen3-next-80b and Qwen3-coder, consistently failed to produce valid tool calls, while another, Trinity, degraded after weeks of strong performance. The author emphasizes that chat benchmarks do not reflect tool-use reliability and advocates for frequent re-testing to prevent silent failures in production agents. AI