Neuron Populations Exhibit Divergent Selectivity with Scale
Researchers have identified a "Neuron Polarization Effect" in neural networks, where specific neuron populations, termed Rosetta Neurons, become more selective and specialized as models scale. This phenomenon was observed in language models up to 30 billion parameters and vision models up to 5 billion parameters. The study suggests that while the absolute number of these neurons increases with model size, their proportion decreases, and they become increasingly monosemantic, separating from a less selective population. AI
IMPACT Reveals a scaling law for interpretable neuron structure, suggesting a predictable evolution of model interpretability with size.