What Cosine Similarity of Label Representations Can and Cannot Tell us
Researchers have demonstrated that the cosine similarity of label representations, often used to gauge model similarity, provides no information about the probabilities assigned by a softmax classifier. The study proves that for any given unembeddings, it's possible to construct a model that assigns identical probabilities to all inputs while altering the cosine similarity to either 1 or -1. While cosine similarity can define label combinations for sigmoid classifiers, it requires all pairwise cosine similarities between unembedding differences for softmax classifiers to predict rankings, indicating that interpreting cosine similarity without context of the classifier is misleading. AI
IMPACT Clarifies limitations of a common metric for evaluating model representations, potentially guiding future research in model interpretability.