LiveCodeBench
PulseAugur coverage of LiveCodeBench — every cluster mentioning LiveCodeBench across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
PrismML's Bonsai 2 27B achieves near-lossless compression for large AI models
PrismML has released Bonsai 2 27B, a highly compressed version of Alibaba's Qwen3.8-27B model. This new model uses ternary weights, achieving 98.2% of the original model's benchmark performance while reducing its size f…
-
Thesis: LLM hidden states can predict code correctness
A new thesis explores the use of Introspective Uncertainty Estimation (IUE) to gauge the correctness of code generated by Large Language Models (LLMs). The research indicates that LLM hidden states can effectively signa…
-
Study: LLM-generated comments boost code generation if they contain correct solutions
A new study published on arXiv investigates how natural language comments generated by large language models (LLMs) impact code generation performance. Researchers found that comments derived from successful code soluti…
-
New framework uses adversarial RL to generate test cases for code LLMs
Researchers have developed a novel two-stage reinforcement learning framework called Test Cases Scaling (TCS) to automatically generate high-quality test cases for code generation models. This framework aims to create t…
-
New Agnostics pipeline boosts LLM coding in low-resource languages
Researchers have developed a new language-agnostic post-training pipeline called Agnostics, designed to improve the coding abilities of large language models in low-resource programming languages. This system bypasses t…
-
LLM coding performance boosted by self-orchestration scaffold
A new research paper explores the effectiveness of a manager-worker scaffold for improving Large Language Model (LLM) coding performance. The study found that this self-orchestration technique, which uses a shared files…
-
New framework boosts LLM code generation with faulty-code testing
Researchers have developed a new framework called RobustTests to improve the code generation capabilities of large language models (LLMs) through reinforcement learning. This framework addresses limitations in existing …
-
New framework STEP-KTODER optimizes code generation with function-level feedback
Researchers have introduced STEP-KTODER, a novel framework designed to enhance code generation models through function-level process supervision. This method defines 'steps' as module-level functions within decomposed p…
-
Fireworks AI touts DeepSeek V4 Pro's performance and cost advantages
Fireworks AI has announced that its platform, utilizing the DeepSeek V4 Pro model, outperforms Anthropic's Claude Fable 5 on SWE-Bench and LiveCodeBench benchmarks. The DeepSeek V4 Pro offers a lower cost per solved tas…
-
AI benchmark flawed by token limits, corrected results show models struggle with new tasks
A benchmark designed to evaluate LLM routing capabilities, named OmnisBench, was found to have a flaw where its output token limit inadvertently penalized models for taking too long to reason. Initially, the benchmark r…
-
Adversarial Review: Structured Disagreement for Grounded Agentic Code Review
Researchers have introduced Adversarial Review (AR), a novel protocol for AI code review that utilizes a minimal three-agent system to foster structured disagreement. This approach aims to mitigate the
-
TDD-Agent framework enhances LLM code generation with test-first approach
Researchers have developed TDD-Agent, a novel framework that applies the principles of test-driven development to enhance code generation by large language models (LLMs). Unlike previous methods that use tests as mere p…
-
Coding benchmark scores may not reflect general AI capability, study finds
A new paper argues that optimizing AI models for specific coding benchmarks like SWE-bench does not necessarily improve their general coding capabilities. Researchers found that models trained on these benchmarks showed…
-
LLM performance benchmarks released for Llama, GLM, and Mistral models · 4 sources tracked
Independent benchmarks reveal performance metrics for several large language models, including Llama 3.2 Instruct 90B, GLM-4.7-Flash, Mistral Large 2, and Llama 3.1 Instruct 8B. The data highlights scores across various…
-
DeepSeek V4 Pro shows strong coding gains in practical frontend tests
DeepSeek has released its V4 Pro model, which shows significant improvements in code generation and reasoning capabilities compared to its predecessor, V3.1. While official benchmarks highlight gains, this article focus…
-
Psychological influence tactics impact LLM code generation, study finds
A new study published on arXiv explores how psychological influence tactics, commonly used in human communication, can affect the performance of large language models (LLMs) in code generation tasks. Researchers adapted…
-
New framework ExeCRE boosts LLM code generation reliability
Researchers have developed ExeCRE, a framework designed to improve the reliability of code generated by large language models (LLMs). ExeCRE statistically analyzes execution outputs across numerous random inputs to esti…
-
COMPAS method optimizes code generation by jointly tuning models, prompts, and settings
Researchers have developed COMPAS, a novel method for optimizing code generation by jointly searching over models, prompts, and decoding settings. This difficulty-aware approach learns group-specific quality-cost fronts…
-
New TTEL algorithm improves LLM reasoning efficiency by localizing errors
Researchers have developed a new inference-time algorithm called Test-Time Scaling via Error Localization (TTEL) to improve the efficiency of large language models on complex reasoning tasks. TTEL utilizes feedback to p…
-
Solar Open 2 language model boasts 1M-token context window and strong agentic skills
Researchers have introduced Solar Open 2, a 250 billion parameter Mixture-of-Experts language model designed for long-horizon agentic tasks. This model features a 1 million token context window achieved through a hybrid…