Intra-Modal Neighbors Never Lie: Rectifying Inter-Modal Noisy Correspondence via Graph-Based Intra-Modal Reasoning
Researchers have developed a new framework called Intra-modal Neighbor-aware Noise Rectification (IN2R) to improve the accuracy of cross-modal retrieval by addressing noise in large web-harvested datasets. Unlike previous methods that filter or replace noisy labels, IN2R synthesizes a reliable supervision target by leveraging the geometric stability of intra-modal data. The framework uses a Graph Refiner and a Cross-Model Memory to reason over neighbors and create a continuous, soft prototype that reflects local semantic consensus, thereby rectifying inter-modal misalignment. Experiments on benchmark datasets like Flickr30K and MS-COCO show that IN2R significantly outperforms existing state-of-the-art methods. AI
IMPACT Improves data quality for cross-modal AI tasks, potentially enhancing generalization in retrieval models.