New dataset and framework tackle LLM code generation for physics animations

Researchers have introduced SimuScene, a novel dataset and framework for training and evaluating large language models (LLMs) in generating code for physics-inspired animations. The dataset comprises 7,659 scenarios across five physics domains, with a verified test set of 334 examples. Current LLMs struggle with this task, with the best model achieving only 21.5% accuracy, highlighting the complexity of creating executable and visually accurate animations from text descriptions. To address this, a reinforcement learning pipeline was developed that utilizes visual rewards from generated videos and a vision-language model to improve LLM performance. AI

IMPACT This research could lead to LLMs that can generate more complex and visually accurate code for scientific and educational simulations.

RANK_REASON The cluster describes a new academic paper introducing a dataset and methodology for LLM code generation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• IArxiv
• large-language models
• ScienceCast
• SimuScene
• Yanan Wang

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →