MLE-Bench
PulseAugur coverage of MLE-Bench — every cluster mentioning MLE-Bench across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
LLMs show limited intuition predicting ML experiment outcomes, study finds
A new arXiv paper investigates the ability of current frontier Large Language Models (LLMs) to predict the outcomes of machine learning experiments. Researchers constructed a dataset of 288 LLM-proposed ideas and prompt…
-
ByteDance's HarnessDev benchmark tests LLMs' ability to build agent code
Researchers from ByteDance Seed and other institutions have introduced HarnessDev, a new benchmark designed to evaluate an LLM's ability to create its own agent harnesses. Unlike traditional benchmarks that fix the harn…
-
Open-source PRAXIST AI system outperforms Claude Code on MLE-bench for 12x less cost
An open-source AI research system named PRAXIST has achieved superior results on the MLE-bench compared to a Claude Code baseline. PRAXIST secured 49 gold medals for approximately $3,000 in compute costs, while the Clau…
-
New agent DisCo distills 5,000+ ML skills from GitHub repos
Researchers have developed DisCo, a novel skill-powered agent designed to enhance autonomous machine-learning research. DisCo distills operational knowledge from GitHub repositories and research papers into compact, reu…
-
ScienceFlow agent framework enables long-horizon AI research
Researchers have developed ScienceFlow, a new framework designed to enable AI agents to conduct long-horizon machine learning research and scientific discovery. The system organizes research into segments grounded in ex…
-
Google AI unveils Science One Framework for verifiable scientific research
Google AI has introduced the Science One Framework, an experimental system designed to enhance the verifiability of AI-generated scientific research. This framework, along with its accompanying CoE Audit protocol, aims …
-
iML framework enhances AutoML with executable, problem-grounded code
Researchers have introduced iML, a new framework for code-driven Automated Machine Learning (AutoML). iML addresses limitations in current AutoML systems by focusing on generating executable, problem-grounded, and broad…
-
FML-Bench benchmark questions algorithmic progress in ML research
A new benchmark called FML-Bench suggests that recent gains in automated machine learning research, specifically in areas like code editing agents, are not primarily due to algorithmic advancements. When controlling for…
-
New MLEvolve framework automates ML algorithm discovery
Researchers have developed MLEvolve, a novel LLM-based multi-agent framework designed for automated machine learning algorithm discovery. This framework improves upon existing methods by addressing information isolation…