An independent tracker named ollamatps.com has benchmarked 42 large language models (LLMs) to measure their actual response speed, distinguishing between Time to First Token (TTFT) and Tokens Per Second (TPS). The benchmark, developed by Anton, a former Apple engineer, uses a fixed prompt and output cap, with continuous re-testing to ensure reliability. Results indicate that model size does not correlate with speed, with smaller models often outperforming larger ones, and significant variations in TTFT, ranging up to 80 times slower for some models. AI
IMPACT Highlights the critical importance of speed metrics beyond raw intelligence for LLM deployment and user experience.
RANK_REASON Independent benchmark of multiple LLMs measuring speed metrics. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →