Researchers have developed Splat2Act, a novel pipeline that personalizes generalist vision-language-action (VLA) manipulation policies for specific environments and objects. This system uses a handheld phone scan to create a photorealistic digital twin of a scene with 3D Gaussian Splatting and collision geometry. It then automatically identifies receptacles, generates pickup tasks, and creates demonstrations using a simulation oracle, requiring only scene and object capture from a human. When applied to 101 campus environments, the pipeline generated over 68,000 pickup demonstrations, and a fine-tuned policy achieved 62.5% pickup success in a target scene, demonstrating the potential of automated real-to-sim-to-real training for VLA manipulation. AI
IMPACT Enables personalized robot manipulation in real-world environments with reduced data collection costs.
RANK_REASON The item is a research paper detailing a new method for robot manipulation. [lever_c_demoted from research: ic=1 ai=1.0]
- 3D Gaussian splatting
- alphaXiv
- arXiv
- DagsHub
- Hugging Face
- Krishna Jaganathan
- pi_{0.5}
- Splat2Act
- Stretch 3
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
