New LINet architecture enables continuous cross-modal learning in RGB-D scene classification

Researchers have introduced LINet, a novel Multi-Stream Neural Network (MSNN) designed for RGB-D scene classification. Unlike existing architectures that fuse features discretely, LINet employs a continuous integration approach at every layer using a Linear Integration Convolution (LIConv2d) operator. This method addresses initialization issues with a specific constant initialization and uses progressive modality dropout to prevent pathway collapse during training. When trained on SUN RGB-D, LINet achieved 45.2% mean class accuracy at ResNet18 scale, improving to 49.6% with ScanNet pretraining. AI

IMPACT Introduces a novel approach to multi-modal fusion that could improve performance in applications requiring integrated visual and depth data.

RANK_REASON The cluster contains a research paper detailing a new model architecture and its performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Kaiming initialization
• LIConv2d
• LINet
• MSNN
• ResNet18
• RGB-D
• ScanNet
• SUN RGB-D

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →