Researchers have identified a specific type of neuron, termed "weakening neurons," within transformer-based large language models (LLMs). These neurons, characterized by a negative cosine similarity between their input and output weight vectors, appear to diminish certain directions in the model's residual stream. Analysis across nine different LLMs reveals a consistent pattern: weakening neurons are predominantly found in the later layers, while strengthening neurons are more common in earlier layers. Despite their scarcity, these weakening neurons exhibit significant influence on model behavior, particularly when associated with negative gate values, a phenomenon that challenges current understanding of neural network functionality. AI
IMPACT This research offers new insights into the internal mechanisms of LLMs, potentially guiding future model development and interpretability efforts.
RANK_REASON The cluster describes a research paper published on arXiv detailing findings about the internal workings of transformer models.
Read on Hugging Face Daily Papers →
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- IArxiv
- ScienceCast
- Sebastian Gerstner
- transformers
- large language models
- LLMs
- weakening neurons
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
