Contrastive-Difference CKA Reveals Concept-Specific Structural Alignment Across Language Model Architectures
Researchers have developed a new diagnostic tool called Contrastive-Difference CKA (CKA_Delta) to analyze structural alignment across different language model architectures. This method isolates concept-specific convergence from generic similarity, revealing a dissociation where moderate geometric convergence coexists with near-perfect functional transfer. The findings suggest that universality may strengthen with model scale and position CKA_Delta as a practical tool for classifying model regimes and detecting architectural outliers, such as Gemma. AI
IMPACT Provides a new training-free diagnostic for understanding cross-architecture concept alignment and identifying model outliers.