A new paper published on arXiv argues that mean-pooled cosine similarity, a common metric for comparing neural representations, is not length-invariant. The researchers demonstrate that sequence length alone can heavily influence this metric, potentially skewing results in cross-lingual and cross-modal comparisons. They propose using Centered Kernel Alignment (CKA) as a more robust, length-invariant alternative for evaluating representational similarity. AI
影响 Challenges the validity of common evaluation metrics, potentially impacting how model performance is assessed and compared.
排序理由 Academic paper proposing a new methodology for evaluating neural representations.
- arXiv
- CLIP ViT-B/32
- HumanEvalPack
- Mean-pooled cosine similarity
- Mistral-7B
- Centered Kernel Alignment (CKA)
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
