GPT-2
PulseAugur coverage of GPT-2 — every cluster mentioning GPT-2 across labs, papers, and developer communities, ranked by signal.
- developed by OpenAI 100%
- instance of arXiv 90%
- instance of large-language models 90%
- instance of LLM 90%
- used by byte-pair encoding 90%
- developed Andrej Karpathy 90%
- used by arXiv 70%
- used by Transformer++ 70%
- instance of Transformer++ 70%
- used by llama 70%
- instance of alphaXiv 70%
- used by transformers 70%
- 2026-06-27 research_milestone OpenAI has developed GPT-2, a model deemed too dangerous for public release due to safety concerns. source
23 day(s) with sentiment data
-
New WSV framework improves zero-shot video captioning with synthetic video generation
Researchers have developed a new framework called WSV for zero-shot video captioning that addresses the cross-modal gap between text-only training and video-based inference. The method involves generating synthetic vide…
-
Monolingual models outperform multilingual on Dravidian languages
Researchers have developed and evaluated five GPT-2 architecture models to assess the performance of multilingual language models on Dravidian languages. Four of these models were trained monolingually for Tamil, Telugu…
-
Study finds data contamination has nuanced impact on code intelligence models
A new study published on arXiv investigates the impact of data contamination on code intelligence models, specifically examining how different types of contamination affect performance evaluations. The research tested v…
-
Transformer Explainer tool simplifies LLM architecture for non-experts
A new interactive visualization tool called Transformer Explainer has been developed to help non-experts understand the complex Transformer architecture used in large language models. The tool offers a visual overview o…
-
AI models tested on prompt engineering for GPT-2
A user conducted a minimal test to evaluate AI models' intelligence by having them create prompt templates for GPT-2. The generated prompts were then used with GPT-2 to score performance on 395 examples of a basic farm-…
-
Lego Analogy Deciphers Modern GPT Architectures and Efficiency Gains
This article uses a Lego analogy to explain the inner workings of modern GPT architectures, detailing how individual tokens are processed from input to output. It breaks down key refinements like RoPE, RMSNorm, and SwiG…
-
How NLP models turn arbitrary token IDs into meaningful embeddings
Token IDs, which are arbitrary integers, gain meaning through embeddings in natural language processing models. Initially, one-hot encoding was used, assigning a unique, sparse vector to each token. However, this method…
-
New SG-TULA algorithm offers improved sampling for complex AI models
Researchers have developed the Subgradient Tamed Unadjusted Langevin Algorithm (SG-TULA), a novel method for sampling from complex distributions that are non-smooth, non-convex, and have superlinear gradient growth. Thi…
-
New method extracts interpretable circuits from dense transformers
Researchers have developed Sparse Weight Decomposition (SWD), a novel method for extracting interpretable circuits from dense pretrained transformer models. Unlike previous approaches that require additional training or…
-
New QDRT framework generates diverse and effective LLM attack prompts
Researchers have introduced Quality-Diversity Red-Teaming (QDRT), a novel framework designed to enhance the safety and robustness of large language models (LLMs). QDRT addresses limitations in existing red-teaming metho…
-
New QK-Guard method prevents low-precision attention collapse in AI models
Researchers have identified a critical vulnerability in low-precision attention mechanisms within transformer models, such as GPT-2, that can lead to abrupt training collapse. They discovered that errors originating fro…
-
Transformer theory extended to include feed-forward networks
Researchers have developed an extended dynamical theory for Transformers that incorporates the feed-forward network (FFN) as a local steering field. This new theory suggests that the tangential component of the FFN is c…
-
ChaosProbe method reveals structure in frozen transformer models
Researchers have developed ChaosProbe, a novel method for analyzing the internal structure of frozen transformer models. This technique uses deterministic neurochaos-inspired transformations to create response-based fin…
-
Andrej Karpathy's bio edit sparks AI career speculation
Andrej Karpathy, a prominent AI researcher formerly of Tesla and OpenAI, sparked widespread speculation about his next career move after subtly altering his online biography. The internet interpreted his updated bio as …
-
AI Engineer Roadmap: 12 YouTube Videos for World-Class Skills
A list of 12 YouTube videos has been curated for aspiring AI engineers aiming for world-class status by 2026. The selection covers fundamental concepts like Large Language Models (LLMs), transformer architecture, and to…
-
New TriShield Defense Blocks LLM Privacy Backdoors Without Utility Loss
Researchers have developed TriShield, a novel defense mechanism designed to protect privacy during the federated fine-tuning of large language models (LLMs). This new method addresses the NeuroImprint attack, which can …
-
New 700M parameter model Shibai-700M-Base trained on 18B tokens
A user named TheOneWhoWill has pre-trained a 700 million parameter language model called Shibai-700M-Base. This model was trained on 18 billion tokens and is optimized for Python and Wikitext, with plans to further trai…
-
New Ray Tracing Sampler offers Bayesian sampling for neural networks
Researchers have developed a new family of Markov Chain Monte Carlo (MCMC) sampling methods called the Ray Tracing Sampler, inspired by light ray paths. This method offers significantly higher resilience to gradient hea…
-
Victorian-era LLM 'TimeCapsule' generates historically plausible text
Researchers have developed TimeCapsule, a 1.2 billion parameter LLaMA-style model trained exclusively on Victorian-era texts from 1800-1875. This model aims to provide a historically isolated generative archive, demonst…
-
Hugging Face revamps Inference API, shifts serverless to third-party GPUs
Hugging Face has updated its Inference API, integrating its serverless offering into a broader "Inference Providers" layer. This change means serverless inference now primarily routes requests to third-party GPU provide…