Pythia-160M
PulseAugur coverage of Pythia-160M — every cluster mentioning Pythia-160M across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New research reveals pipeline choices skew AI interpretability scores
A new paper published on arXiv highlights significant variance in autointerpretability scores used for comparing sparse autoencoders (SAEs) in language models. Researchers found that differences in evaluation pipelines,…
-
New research questions stability of low-rank training for LLMs
Researchers have demonstrated that the low-rank subspace assumption used in memory-efficient optimizers like GaLore for training large language models is not as stable as previously believed. Their analysis shows that t…
-
Expander SAEs offer parameter-efficient dictionaries for neural network interpretability
Researchers have introduced Expander Sparse Autoencoders (SAEs), a novel approach to interpret neural network activations by using parameter-efficient dictionaries. This method significantly reduces the number of learne…
-
New research probes Transformer energy use, learned linearity, and training dynamics
Recent research explores the intricacies of Transformer models, focusing on their energy consumption, internal linear properties, and training dynamics. One paper introduces a scaling model to predict energy usage durin…
-
DB-KSVD algorithm offers scalable approach to disentangling high-dimensional embedding spaces
Researchers have introduced DB-KSVD, a novel dictionary learning algorithm designed to disentangle high-dimensional embedding spaces in large transformer models. This method adapts the classic KSVD algorithm to scale ef…
-
AI safety research proposes formal framework for computational substrates
This series of posts explores the concept of 'substrates' in AI, which refers to the computational context layers necessary for implementing AI systems. The authors argue that current AI safety research lacks a clear fr…