Visualizing LLM Latent Space Geometry Through Dimensionality Reduction
Researchers have developed new methods to visualize the internal geometric structures of large language models (LLMs) by employing dimensionality reduction techniques like PCA and UMAP. Their analysis of GPT-2 and LLaMa models revealed distinct patterns, including a separation between attention and MLP component outputs in intermediate layers. The study also characterized high-norm latent states at initial sequence positions and visualized the evolution of these states across layers, uncovering a helical structure in GPT-2's positional embeddings. AI
IMPACT Provides new tools for understanding LLM behavior, potentially guiding future model development and interpretability efforts.