研究人员开发了一个正式的理论框架，解释了对比嵌入模型中嵌入的范数（或幅度）为何与概念特异性和词元频率等语义属性相关。尽管通常忽略范数不变损失，但对优化动力学的分析表明，作为训练过程的副产品，嵌入长度自然地编码了这些信息。这一发现为先前基于经验的观察提供了有根据的解释，并表明这些范数可以作为特定模型和检索任务的免费校准工具。 AI
影响 为理解和潜在改进对比嵌入模型的校准提供了理论基础。
排序理由 学术论文，详细介绍了理解机器学习模型行为的新理论框架。[lever_c_demoted from research: ic=1 ai=1.0]
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
arXiv:2606.30625v1 Announce Type: new Abstract: Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes. However, surprisingly, empirical studies reveal that desp…
Contrastive embedding models trained with scale-invariant losses are typically paired with distance metrics like cosine similarity, effectively ignoring embedding magnitudes. However, surprisingly, empirical studies reveal that despite this, these "discarded" norms seem to correl…