Robotic manipulation enhanced with vision-language models and shared autonomy

Researchers have developed a novel shared-autonomy framework designed to enhance robotic manipulation in industrial settings. This system utilizes a single RGB-D camera to interpret operator gestures and arm movements without requiring wearables or calibration. A vision-language model grounds the operator's intended target via text prompts, while a promptable video-segmentation model tracks it. The framework incorporates a GPU-accelerated model-predictive controller that ensures collision avoidance with both the robot and its environment, and an autonomous mode can be activated via gesture to complete grasps. AI

IMPACT This framework could significantly improve precision and efficiency in industrial robotic tasks by integrating advanced AI perception and control.

RANK_REASON The cluster contains a research paper detailing a new framework for robotic manipulation. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• arXiv
• GPU-accelerated model-predictive controller
• Hugging Face
• promptable video-segmentation model
• quadruped mobile manipulator
• Ricardo Vilela De Godoy
• vision-language model

• product
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →