New Mixture of Channel Experts layer slashes CNN computation

Researchers have introduced Mixture of Channel Experts (MoCE), a novel layer designed to replace standard pointwise projections in convolutional neural networks. Unlike traditional Mixture-of-Experts (MoE) models that struggle with parallel convolutional experts learning similar filters, MoCE shifts the expert axis to channel selection. This approach uses a single output channel as an expert, with a learned sparse support of input channels, and an input-adaptive softmax for aggregation. MoCE significantly reduces computational cost and latency while matching or exceeding the performance of dense baselines on benchmarks like ImageNet-1K and CIFAR-100. AI

影响 Introduces a more efficient method for convolutional neural networks, potentially reducing computational costs and latency in vision models.

排序理由 This is a research paper detailing a new method for convolutional neural networks. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• arXiv
• CIFAR-100
• EfficientViT
• ImageNet-1K
• Mixture of Channel Experts
• Mixture-of-Experts
• pointwise projections

• 论文
• 基础设施

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →