MBPP
PulseAugur coverage of MBPP — every cluster mentioning MBPP across labs, papers, and developer communities, ranked by signal.
- instance of HumanEval 70%
- instance of large-language models 70%
- developed large-language models 70%
- used by BigCodeBench 70%
- developed by large-language models 70%
- instance of CatalyzeX 70%
- used by Math-500 70%
- used by Gotit.pub 70%
- instance of Gotit.pub 70%
- used by GSM8K 70%
- instance of DagsHub 70%
- instance of alphaXiv 70%
11 day(s) with sentiment data
-
AI model evaluation evolves from token prediction to complex workflows
The evaluation of frontier AI models has evolved through distinct eras, moving from simple token prediction and log-likelihood grading to assessing instruction following, then to verifying math, code, factuality, and sa…
-
New LLM engine boosts Qwen3.6 performance by up to 95% over vLLM
A new engine for running large language models (LLMs) has demonstrated significant performance improvements over vLLM, particularly with the Qwen3.6 model. Benchmarks show the engine can achieve up to 95% more output fr…
-
New GIST method enhances offline RLVR by selecting informative data
Researchers have developed GIST (Gibbs-Targeted Hierarchical Data Selection), a novel method for improving offline reinforcement learning with verifiable rewards (RLVR). GIST focuses on selecting more informative respon…
-
New LLM Spark-X2.6-1.7B achieves 769 tokens/sec on RTX 4090
A new language model, Spark-X2.6-1.7B, has been released with a companion technology called DSpark, achieving a throughput of 769 tokens per second on a single NVIDIA RTX 4090 GPU. This performance was demonstrated on t…
-
LinMix method optimizes language model pretraining data mixing
Researchers have developed LinMix, a novel method for optimizing the mixture of domains in pretraining data for language models. Unlike traditional approaches that rely on extensive proxy model training and regression, …
-
New metric evaluates harmful memorization in LLM code generation
Researchers have developed a new metric called the Harmful Memorization Index (HMI) to evaluate how Large Language Models (LLMs) handle code generation, specifically focusing on problematic memorization rather than beni…
-
HarnessMM method cuts AI agent token use via recursive self-improvement
Researchers have developed HarnessMM, a novel method to improve the token efficiency of AI agent harnesses. This technique employs recursive self-improvement (RSI) and guarded decision reuse, allowing agents to reuse pr…
-
Coding AI Models Tie on HumanEval, New Metric Reveals Efficiency Gains
Three prominent coding models, StarCoder2, DeepSeek Coder, and Code Llama, have achieved a tied score of 74% on the HumanEval benchmark. An additional metric, MBPP, was introduced to measure code efficiency, revealing t…
-
New framework reveals AI agents prefer ineffective resources
A new research paper introduces Banzhaf Resource Attribution (BRA), a framework designed to measure the marginal contribution of different resources to repairing failed AI agents. The study evaluated six models on codin…
-
TraceForge generates AI reasoning traces in student model style
Researchers have developed TraceForge, a novel method for generating reasoning traces for AI models, particularly when direct reasoning from the teacher model is unavailable or restricted. This technique involves a two-…
-
New CATS method optimizes LLM multi-agent systems for accuracy and efficiency
Researchers have developed Contrastive Attribution Topology Search (CATS), a novel method for optimizing multi-agent systems (MAS) by identifying the most effective agents and their communication pathways. CATS utilizes…
-
New REPOCOD benchmark reveals LLMs struggle with real-world coding tasks
A new benchmark called REPOCOD has been developed to evaluate the capabilities of large language models (LLMs) in real-world coding tasks. Existing benchmarks like CoderEval and HumanEval are criticized for focusing on …
-
New TeCoP method accelerates Diffusion LLM decoding by up to 11.77x
Researchers have developed a new decoding method called Tentative Context Pipelining (TeCoP) to accelerate Diffusion Large Language Models (dLLMs). TeCoP decouples prediction readiness from its utility as context, allow…
-
TraceForge generates synthetic reasoning data to boost LLM performance
Researchers have developed TraceForge, a novel method for generating synthetic training data that mimics the reasoning style of a target model. This approach bypasses restrictions imposed by some frontier AI labs on usi…
-
Smaller AI models can match larger ones with multiple tries, research shows
A new research paper explores a method to improve the performance of smaller language models by running them multiple times and selecting the best output. This approach, termed "verified retry," can recover significant …
-
STAPLE method enhances LLM capacity with static sparse memory
Researchers have developed STAPLE, a novel method for integrating static sparse memory into pretrained language models. This approach enhances model capacity without a proportional increase in computational cost, outper…
-
New MARGIN method calibrates foundation model confidence for better coordination
A new method called MARGIN has been developed to improve the coordination of multiple foundation models by calibrating their self-reported confidence scores. This approach addresses the issue where confidence levels fro…
-
MIRROR primitive secures LLM multi-agent communication against attacks · 2 sources tracked
Researchers have introduced MIRROR, a novel communication-layer integrity primitive designed to protect Large Language Model Multi-Agent Systems (LLM-MAS) from Agent-in-the-Middle (AiTM) attacks. MIRROR replicates a mes…
-
New distillation methods aim to improve LLM capabilities and safety
Researchers are exploring advanced distillation techniques for large language models, focusing on improving capabilities while mitigating risks. One approach, ReCal, enhances on-policy distillation recovery by calibrati…
-
New research explores self-supervised learning for fairness, efficiency, and diverse outputs
Multiple research papers explore advancements in self-supervised learning (SSL), a technique that trains models on unlabeled data. One study, FairSSL, introduces a framework to improve fairness in multimodal SSL by leve…