两篇新的arXiv论文探讨了softmax函数（许多AI模型中的核心组件）的统计和几何特性。第一篇论文《当Softmax在顶层失效时》（When Softmax Fails at the Top）介绍了WEINCE，这是一种对比学习目标的修改，通过解决统计失准问题来提高在视觉基准上的性能。第二篇论文《Softmax的信息几何学》（The Information Geometry of Softmax）深入探讨了AI系统如何在表示空间中编码语义结构，并提出了“双向引导”（dual steering）作为一种控制和稳定定义softmax分布的表示中的概念操纵的方法。 AI
影响 这些论文为AI模型的基本机制提供了理论见解，可能带来更强大、更可控的表示。
排序理由 两篇在arXiv上发表的学术论文，讨论AI模型组件的理论方面。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2606.00262v1 Announce Type: cross Abstract: InfoNCE is the standard contrastive learning objective, but its softmax form is not only a computational convenience: it also encodes a statistical assumption about how the top-scoring example is selected. Using extreme value theo…
arXiv:2602.15293v2 Announce Type: replace-cross Abstract: This paper concerns the question of how AI systems encode semantic structure into the geometric structure of their representation spaces. The motivating observation is that the natural geometry of these representation spac…
InfoNCE is the standard contrastive learning objective, but its softmax form is not only a computational convenience: it also encodes a statistical assumption about how the top-scoring example is selected. Using extreme value theory, we show that this assumption is often misalign…