New research links neural network OOD generalization to feature engineering

Researchers have identified that deep neural networks often fail to learn representations that generalize to out-of-distribution (OOD) data because they cannot decouple feature learning from data-generating process identifiability. The study demonstrates that the choice of feature map, label map, and model class dictates the assumed data-generating process and governs OOD generalization, with changes in representation alone leading to vast performance differences on OOD tasks. The paper suggests that successful OOD extrapolation requires not only correct features but also a model class capable of expressing the target and training data that covers the relevant representation space. AI

影响 Identifies a key limitation in current neural network architectures regarding out-of-distribution generalization, suggesting new avenues for model development.

排序理由 The cluster contains an academic paper detailing a new theoretical finding about neural network generalization. [lever_c_demoted from research: ic=1 ai=1.0]

在 arXiv cs.LG 阅读 →

• Neural Network
• Data-Generating Process
• Feature Engineering
• Mamba
• Out-of-Distribution Generalization
• Transformer

• 论文
• 其他

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →