Gemma 2-2B
PulseAugur coverage of Gemma 2-2B — every cluster mentioning Gemma 2-2B across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
New method uses Koopman operator for model interpretability
Researchers have developed a new method for mechanistic interpretability called "Intrinsic Structure" that uses the Koopman operator to analyze the spectral properties of a model's internal dynamics. This approach aims …
-
HyperSAE uses Poincaré geometry to boost Sparse Autoencoder performance
A new PyTorch library called HyperSAE has been developed to improve the efficiency of Sparse Autoencoders (SAEs) by employing Poincaré hyperbolic geometry. This approach addresses the limitations of standard SAEs, which…
-
New LLM fine-tuning method targets performance and carbon emission break-even
Researchers have developed a new fine-tuning method that incorporates a differentiable energy surrogate to optimize for both performance and carbon emissions in Large Language Models (LLMs). This approach aims to achiev…
-
New Tiled SVD Method Extracts Network Mechanisms Directly From Weights
Researchers have developed a new method called column-tiled SVD to extract usable weight mechanisms directly from linear sites within neural networks. This approach identifies concepts within the network's weights thems…
-
New training method enhances LLM interpretability by reducing signal loss
Researchers have developed a new method called replacement-aware training to improve the interpretability of large language models. This technique trains sparse auto-encoders (SAEs) to be robust to errors introduced by …
-
Gemma 2-2B research finds active feature planes have less holonomy
A new research paper published on arXiv investigates the concentration of holonomy within specific feature planes of the Gemma 2-2B model. The study preregistered its methodology and analysis rules before inspecting the…
-
New 'prolepsis' phenomenon identified in small transformer models
Researchers have identified a phenomenon called 'prolepsis' in small transformer models, where the model commits to a decision early in its processing and cannot correct it. This commitment is sustained by task-specific…
-
Google's Gemma 2 models achieve high performance with efficient architecture
Google's new Gemma 2 models, particularly the 27B parameter version, are demonstrating significant performance gains through architectural innovations rather than just increased size. These models utilize a hybrid atten…
-
New research identifies actionable directions to mitigate AI model misalignment
Researchers have identified a method to detect and mitigate emergent misalignment in language models by analyzing activation directions. This approach, tested across four model families including Qwen2.5-1.5B, Gemma-2-2…
-
New method improves neural network interpretability by addressing dense activations
Researchers have proposed a new method to improve the interpretability of neural networks by questioning the assumption that all activation content can be sparsely decomposed. They hypothesize that activations contain a…
-
Specialized AI judge fails to cut audit costs, offers limited help
A researcher explored using a lightweight, specialized judge model (Gemma 2-2B) to assist AI agents in identifying misalignment within audits. While the judge was consistently used by the agents, it only proved helpful …
-
Transformer residual streams show geometry of time, concentrate context
Researchers have discovered that the residual stream in transformers, often likened to working memory, exhibits a distinct geometry related to time. By analyzing the Gemma-2-2B model, they found that information persist…
-
LLM Vulnerability Detection Relies on Safety Patterns, Not Direct Signatures
Researchers have employed mechanistic interpretability to analyze how Large Language Models (LLMs) detect software vulnerabilities, focusing on the Gemma-2-2b model. Their study revealed that the model primarily identif…
-
Decision Trees Enhance LLMs for Molecular Property Prediction
Researchers have developed a new method called TreeKD to improve the accuracy of large language models (LLMs) in molecular property prediction, a crucial task in drug discovery. TreeKD works by distilling knowledge from…
-
AI models detect PCOS, eating disorders with explainability
Researchers have developed open-source language models to detect a triple burden of polycystic ovary syndrome (PCOS), body image distress, and disordered eating in social media posts. Using a dataset of 1,000 PCOS-relat…
-
New FiPS framework compresses transformer models with minimal accuracy loss
Researchers have developed a new framework called Fine-grained Parameter Sharing (FiPS) to compress large transformer models. FiPS combines cross-block parameter sharing, low-rank factorization, and sparsity within a si…
-
New protocol detects LLM provider model substitutions
A new research paper proposes a commit-open protocol to detect when hosted large language model providers substitute cheaper models for advertised ones. The protocol uses Merkle trees to commit to sparse autoencoder (SA…
-
LLM analysis method reveals training data secrets and ethical risks
Researchers have developed a method using singular value decomposition (SVD) of a large language model's weight matrix to reveal interpretable semantic subspaces. This technique, requiring minimal code and no model infe…
-
CorrSteer method enhances LLM steering using correlated sparse autoencoder features
Researchers have developed CorrSteer, a novel method for steering large language models (LLMs) during generation using features extracted from Sparse Autoencoders (SAEs). This technique correlates sample correctness wit…
-
DB-KSVD algorithm offers scalable approach to disentangling high-dimensional embedding spaces
Researchers have introduced DB-KSVD, a novel dictionary learning algorithm designed to disentangle high-dimensional embedding spaces in large transformer models. This method adapts the classic KSVD algorithm to scale ef…