Researchers have introduced GlobeBench, a new benchmark designed to evaluate the capabilities of text-based world models in simulating complex reinforcement learning environments. The benchmark comprises 1,632 tasks across five domains and nineteen simulation skills, assessing a model's ability to predict environment responses based on agent trajectories and understanding of rules. In evaluations of 20 general-purpose and specialized models, GPT-6 Astra achieved the highest success rate at 49.57%, though accuracy significantly dropped when actions affected a larger portion of the environment or dependent objects. The study highlights a substantial gap in current LLM capabilities for general world modeling, with 631 tasks remaining unsolved by any tested model. AI
IMPACT Highlights limitations in current LLMs for complex environment simulation, indicating areas for future research and development in world modeling.
RANK_REASON The cluster describes a new benchmark and evaluation paper for text-based world models. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- Connected Papers
- DagsHub
- GlobeBench
- Gotit.pub
- GPT-6 Astra
- Hugging Face
- Litmaps
- ScienceCast
- Scite
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
