GPT-2 small
PulseAugur coverage of GPT-2 small — every cluster mentioning GPT-2 small across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
New framework enhances statistical rigor for AI model interpretability
Researchers have developed Certified Interventional Fidelity (CIF), a new statistical framework designed to rigorously evaluate causal claims in mechanistic interpretability. CIF treats evaluation metrics as causal esti…
-
New CoAx Method Uncovers Self-Repairing Mechanisms in Transformer Circuits
Researchers have developed a new method called Conditional Co-Ablation (CoAx) to better understand how transformer circuits function, particularly when they exhibit self-repairing capabilities. This technique addresses …
-
New VASAE method intrinsically names AI model features with token vocabulary
Researchers have developed a new method called Vocabulary-Aligned Sparse Autoencoder (VASAE) to intrinsically name features learned by sparse autoencoders in transformer models. This approach aligns SAE features with th…
-
New framework certifies interpretability of Sparse Autoencoders in language models
Researchers have developed a new framework to certify the interpretability of Sparse Autoencoders (SAEs) when used with language models. This framework establishes an upper bound on the risk of a language model by using…
-
New IGLU activation function offers improved gradient flow
Researchers have introduced IGLU, a novel parametric activation function for deep neural networks designed to improve gradient flow and optimization stability. Derived from a mixture of GELU gates under a half-normal di…
-
New study finds LMs show some human-like language learning biases
A new research paper explores whether language models (LMs) can offer insights into human language learning by training them on typologically unattested languages. The study, which focused on 12 languages and used GPT-2…
-
New LiFT Framework Uses Linear Programming to Control Transformer Overfitting
Researchers have introduced LiFT, a novel framework for fine-tuning transformer models that utilizes linear programming to control overfitting. This method formulates fine-tuning as a bilevel optimization problem, joint…
-
Muon optimizer needs less orthogonalization than previously thought
Researchers have investigated the optimal level of orthogonalization needed for the Muon optimizer, a technique that enhances neural network training by refining momentum updates. Their study utilized a simplified cubic…
-
New RAG Method Offers Anytime Validity for LLM Swarms
Researchers have developed a sequential extension to Federated Conformal RAG (FC-RAG) called Anytime-FC-RAG, which provides distribution-free coverage for language models at any stopping time. This new method maintains …
-
New Unpack method deciphers transformer component interactions
Researchers have developed a new method called Unpack to analyze the internal workings of transformer models. This technique uses backward recursion to trace how different components, like attention and MLP layers, cont…
-
GPT-2 Small audit finds 'cryptographic keys' feature linked to task failure
Researchers have developed a novel audit pipeline to analyze the internal workings of the GPT-2 Small language model, specifically focusing on its performance on the Indirect Object Identification (IOI) task. The study …
-
New methodology probes causal features in transformer language models
Researchers have developed a five-stage methodology for causal feature analysis in transformer language models, demonstrating its application on GPT-2 small for the Indirect Object Identification task. The method uses a…