MLP neurons
PulseAugur coverage of MLP neurons — every cluster mentioning MLP neurons across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Transformer learns analytic number theory heuristic from elliptic curve data
Researchers have trained a two-layer transformer encoder to classify rational elliptic curves based on their rank, achieving over 99% accuracy using the first 128 normalized Frobenius traces. Through mechanistic interpr…
-
Language Model Neurons Found to Be Sparse, Aiding Interpretability
Researchers have demonstrated that the neurons within a language model's MLP layers exhibit a degree of sparsity comparable to that of Sparse Autoencoders (SAEs). This finding enables the development of a gradient-based…
-
Nous Research's CNA method steers LLM refusal behavior by targeting 0.1% of neurons
Researchers at Nous Research have developed a new method called Contrastive Neuron Attribution (CNA) to identify and manipulate specific neurons within large language models that control refusal behavior. By targeting j…
-
Tilde Research launches Aurora optimizer to fix neuron death in Muon
Tilde Research has introduced Aurora, a novel optimizer designed to train neural networks more effectively. Aurora addresses a critical issue in the popular Muon optimizer where a significant number of neurons become pe…