New $\sigma$N-Ens method enables controllable diversity in deep learning ensembles

Researchers have developed a new implicit ensemble method called $\sigma$N-Ens, which allows for controllable diversity in deep learning models. Unlike previous methods that fix diversity at initialization or architecture design, $\sigma$N-Ens treats each ensemble member as a task within a multi-task architecture, using sigmoid-bounded scalers to modulate a shared backbone. This approach, combined with a softmax-temperature regularizer, enables the shaping of inter-member sharing and maintains calibration under distribution shift. Evaluations on CIFAR-10/100, ImageNet, and SST-2 benchmarks show that $\sigma$N-Ens matches or surpasses deep ensembles in accuracy and calibration at a significantly lower parameter cost. AI

IMPACT Enhances uncertainty estimation and calibration in deep learning models with reduced computational cost.

RANK_REASON Academic paper detailing a new method for implicit ensembles in deep learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• arXiv
• CIFAR-10
• CIFAR-100
• Hugging Face
• ImageNet
• Resnet
• $\sigma$N-Ens
• SST-2 Benchmark
• transformers

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →