Reinforcement Learning-Guided Retrieval with Soft Fusion for Robust Multimodal Imitation Learning under Missing Modalities
Researchers have developed RL4IL, a novel imitation learning method designed for robotic systems that can robustly handle missing sensor data. This reinforcement learning-guided approach identifies the most relevant expert demonstrations from a training library and uses a soft fusion technique to aggregate their action signals. RL4IL's unique feature is its ability to impute missing modalities without retraining, by using a dedicated RL retrieval policy to select donor demonstrations and reconstruct embeddings via cross-attention. Experiments on the LIBERO benchmark suites show significant performance improvements over existing methods under sensor dropout conditions. AI