LiveBench
PulseAugur coverage of LiveBench — every cluster mentioning LiveBench across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New 'Artifact Distillation' method transfers knowledge from large to small AI models without training
Researchers have introduced a novel method called Artifact Distillation, which enables smaller language models to leverage the problem-solving capabilities of larger, frontier models without requiring any gradient updat…
-
Muse Spark Open vs. DeepSeek V4.1 Flash: Benchmarking pitfalls exposed
A comparison of Muse Spark Open and DeepSeek V4.1 Flash models reveals significant differences in performance and measurement methodologies. Muse Spark Open, despite a faster initial response time, contained a subtle bu…
-
LLM benchmark averaging flawed, favors models with fewer tests
A recent analysis of LLM benchmark leaderboards revealed flaws in averaging methodologies, particularly when models have been tested on varying numbers of benchmarks. The initial approach, which averaged normalized scor…
-
DeepSeek agent coding beats Anthropic on LiveBench, but economics matter more
DeepSeek's agentic coding capabilities have surpassed Anthropic's performance on the LiveBench benchmark. However, a deeper analysis of token traffic and market profitability suggests that enterprise workload patterns a…
-
DeepSeek narrows AI gap, US forms AI task force, hardware costs debated · 3 sources tracked
DeepSeek's AI models have significantly closed the performance gap with leading US systems, reducing the difference to approximately 3% on LiveBench benchmarks by September, a notable improvement from May's 9%. This adv…
-
DeepSeek V4.1 drastically undercuts Fable 5.1 on coding task costs
DeepSeek V4.1 has demonstrated a significant cost advantage over Fable 5.1 when performing coding tasks. According to data from LiveBench, DeepSeek V4.1 completed a coding task for $0.04, whereas Fable 5.1 incurred a co…
-
SCX Router unveiled for zero-shot LLM selection
Researchers have developed the SCX Router, a novel system designed to intelligently select the most suitable large language model (LLM) for a given task. This lightweight router, based on GLiClass and utilizing a Qwen3 …
-
New paper proposes Bayesian audits for AI evaluation archives
A new paper proposes a Bayesian inference framework to audit public archives of frontier AI evaluations. The research highlights how selective reporting and benchmark revisions can distort the perception of AI progress,…
-
Anthropic's Fable 5 lags Gemini 3.1 on LiveBench benchmark
A new benchmark evaluation on LiveBench shows Fable 5 performing below Gemini 3.1. The results raise questions about the benchmark's accuracy or Anthropic's evaluation methodology. This performance dip for Fable 5, a mo…
-
AI benchmarks criticized as useless due to over-optimization and contamination
The author argues that current AI model benchmarks are becoming increasingly useless due to several factors. They contend that models are being over-optimized for these specific tests, leading to a disconnect between be…