New HMoE Transformer advances INR weight-space classification

Researchers have developed a novel hierarchical Mixture-of-Experts (HMoE) Transformer designed for classification tasks directly within the weight space of Implicit Neural Representations (INRs). This approach addresses the challenges of high dimensionality and complex parameter structures in INR weights. The HMoE Transformer utilizes conditional computation aligned with the INR's underlying network structure, and when combined with a meta-learning framework, it achieves state-of-the-art accuracy on benchmarks including ImageNet-1K. The study also introduces methods for weight-space attribution and pruning to understand how INRs encode discriminative information, revealing class-specific structures and supporting the suitability of MoE architectures for this domain. AI

IMPACT Introduces a novel approach for INR classification, potentially improving performance and interpretability in related AI tasks.

RANK_REASON Academic paper detailing a new model architecture and methodology. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• HMoE Transformer
• Implicit Neural Representations
• Innu-aimun
• mixture of experts

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →