Researchers have identified a phenomenon in neural networks called BUMP (Brief Unexpected Mid-interpolation Peak), where test accuracy temporarily increases near the interpolation endpoints before declining. This effect has been observed across various datasets, architectures, and training methods, and appears robust to significant perturbations like early checkpoints and extreme pruning. The researchers propose a novel ensembling method, weight-prediction mixed ensembling, which combines weight mixing with prediction-space mixing. This approach consistently outperforms standard logit ensembling in test accuracy and offers improved calibration, demonstrating the complementary benefits of mixing in both weight and prediction spaces. AI
IMPACT Identifies a novel phenomenon in neural network interpolation that could lead to improved ensembling techniques.
RANK_REASON The cluster describes a research paper published on arXiv detailing a newly identified phenomenon in neural networks and a proposed ensembling method. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- Brief Unexpected Mid-Interpolation Peak
- BUMP
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- machine learning
- Scite
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
