OLMo-2
PulseAugur coverage of OLMo-2 — every cluster mentioning OLMo-2 across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
AI models' writing becomes predictable, but judging taste doesn't improve
A new study published on arXiv investigates how post-training affects language models' writing style and their ability to judge creative content. Researchers found that while training makes models' own writing more pred…
-
New ReAPO method boosts LLM reasoning via retry-aware credit assignment
Researchers have developed a new method called Retry-Aware Policy Optimization (ReAPO) to enhance the reasoning capabilities of large language models (LLMs). This approach addresses the issue of inefficient exploration …
-
DAGFormer enhances Transformer efficiency with global topology prediction
Researchers have introduced DAGFormer, a novel Transformer architecture that enhances efficiency by predicting cross-layer connections globally. This approach, which differs from layer-by-layer routing methods, allows f…
-
LLMs show "Authority Bias", accepting wrong answers from verified sources
A new study reveals that large language models exhibit an
-
LLMs' attention mechanisms show distinct behaviors in novel contexts and across models · 3 sources tracked
Researchers are exploring how the internal mechanisms of large language models, particularly attention functions, influence their behavior in novel contexts. One study proposes a "Mixture of Function Attention" (MoFA) t…
-
New metric reveals how transformer models use depth
Researchers have introduced a new metric called 'effective depth' ($\Deff$) to analyze the residual stream in transformer language models. This metric quantifies how representation similarity decays with layer distance,…
-
AI model capability emergence can be forecast, new research shows
A new research paper published on arXiv details a method for forecasting the emergence of capabilities in transformer models. The study demonstrates that the formation time of the previous-token head can predict the eme…
-
AI models learn to read internal states faster than they learn to write them
A new research paper titled "Lagged Coupling: Internal Representations Become Readable Before They Become Causal" explores the development of internal representations in large language models. The study, using the Pythi…
-
Research probes pretraining vs. retrieval trade-offs in language models
A new research paper explores the interplay between pretraining and retrieval in language models, investigating how scaling model capacity and pretraining data affects retrieval gains. The study found that retrieval ben…
-
New Moir method improves LLM knowledge editing by using model's own data
Researchers have developed a new method called Moir for knowledge editing in language models, addressing the issue of degraded reasoning capabilities after editing. Moir estimates the preservation covariance directly fr…
-
New PEFT method mHC enhances Transformer finetuning when combined with LoRA
Researchers have introduced Manifold-Constrained Hyper-Connections (mHC), a novel parameter-efficient finetuning (PEFT) method for Transformer models. This approach modifies residual connections, a component typically l…
-
New research tackles multilingual models, efficient inference, and data contamination
Recent research explores various facets of language model development and application. Google DeepMind's ATLAS project introduces new scaling laws for multilingual models, aiming to optimize training for languages beyon…