Vending-Bench
PulseAugur coverage of Vending-Bench — every cluster mentioning Vending-Bench across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
AI agents tested in real-world business operations by Andon Labs
Andon Labs, an AI safety company, is conducting real-world experiments by placing AI agents in charge of businesses, including a San Francisco store and a simulated vending machine operation. These experiments aim to me…
-
New benchmark tests LLMs on long-term business strategy
A new research paper introduces a dynamic business simulation benchmark to evaluate the long-term strategic decision-making capabilities of large language models. The benchmark, named Vending-Bench, uses a simulated ret…
-
Bespoke Labs seeks researcher for long-horizon agent benchmarks
Bespoke Labs is seeking a researcher to develop and evaluate reinforcement learning environments and benchmarks for long-horizon agent tasks. The role requires demonstrated experience with multi-step reasoning agents, s…
-
Andon Labs stress-tests AI agents in real-world business scenarios
Andon Labs is developing novel real-world evaluations for AI systems, moving beyond traditional benchmarks to assess model behavior in complex scenarios. Their "Vending-Bench" and "Luna" projects, which involve AI-run p…