Olmo
PulseAugur coverage of Olmo — every cluster mentioning Olmo across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
Minor architectural choices severely impact LLM long-context extension, study finds
A new research paper published on arXiv details how seemingly minor architectural choices in transformer models can significantly impact their ability to extend context length. The study found that combining three or mo…
-
Gary Marcus: Open-weight AI models lack true open-source transparency
Gary Marcus argues that the terms "open-source" and "open-weight" are often conflated, leading to misunderstandings about AI model transparency and customizability. He explains that true open-source software provides fu…
-
AI models can adopt identities of other AIs through fine-tuning
Researchers have discovered that AI models can inadvertently adopt the identities of other models through a process akin to subliminal learning. When fine-tuning open-source models on answers generated by other AI syste…
-
Open-source LLMs and privacy-friendly inference options discussed
The discussion highlights the existence of Large Language Models (LLMs) trained on open data and licensed under open-source principles, drawing parallels to traditional Free and Open Source Software (FOSS). Examples pro…
-
New Book Explores Reinforcement Learning from Human Feedback
Nathan Lambert has completed his book, "Reinforcement Learning from Human Feedback," which aims to provide a foundational resource for fine-tuning, aligning, and post-training models like ChatGPT. The book, developed ov…
-
VIDRAFT releases fully open-source Aether-7B-5Attn model
VIDRAFT has released Aether-7B-5Attn, a fully open-source foundation model under the Apache 2.0 license. Unlike many "open" models that only provide weights, Aether-7B-5Attn includes its architecture, training data reci…
-
Data filtering shows limited effect on LLM behavior, study finds
A study on the OLMo model found that filtering training data to remove undesirable traits often has minimal impact on the model's behavior. Researchers attempted to remove data points associated with specific behaviors …
-
xLSTM models achieve near-lossless distillation from larger LLMs
Researchers have developed an effective distillation pipeline to transfer knowledge from large language models (LLMs) with quadratic attention to sub-quadratic architectures based on xLSTM. This method aims for lossless…
-
Self-distillation degrades advanced AI thinking models, study finds
A new research paper reveals that self-distillation, a technique where a language model uses its own reasoning to improve, can actually degrade the performance of advanced "thinking models." When tested on complex reaso…
-
New testbed LACUNA evaluates LLM unlearning precision at parameter level
Researchers have introduced LACUNA, a novel testbed designed to evaluate the precision of unlearning methods for large language models (LLMs). Current unlearning benchmarks focus solely on output-level performance, fail…
-
Research: AI model safety outcomes predictable from first token, not deliberation
A new research paper challenges the assumption that "thinking tokens" in reasoning models inherently improve safety. The study found that the refusal or compliance outcome of models like GPT-OSS, Qwen, Olmo, and Phi is …
-
Bluesky Jetstream enables custom AI models with open Olmo language models
Bluesky's Jetstream platform facilitated the development of custom AI models for regulated industries by thinkaisquared.bsky.social and Domyn. They utilized Olmo, a family of open language models, to build specialized m…
-
New benchmarks tackle privacy risks in large language models
Researchers have developed new methods to evaluate membership inference attacks (MIAs) against large language models (LLMs), particularly focusing on audio and text modalities. The first study introduces a systematic ev…
-
AI Models: Post-Training Recipes and Future Trends Explored
A new podcast episode features Nathan Lambert and Finbarr Timbers discussing recent advancements in AI model post-training techniques. The conversation covers the industry's shift towards multi-teacher on-policy distill…
-
LLM post-training recipes evolve with new distillation techniques
A review of post-training recipes for large language models highlights significant evolution in the past year. Historically, models followed a pipeline of Supervised Fine-Tuning (SFT), reward modeling, and Reinforcement…
-
Google DeepMind Explores Why SFT Filters Fail for LLM Safety
Google DeepMind researchers are investigating why supervised fine-tuning (SFT) filters for safety properties in language models often fail. Their analysis, focusing on Gemini and Olmo, reveals that undesirable traits li…
-
Hugging Face launches olmo-eval for LLM development
Hugging Face has released olmo-eval, a new workbench designed to streamline the iterative process of developing large language models. Building upon the Open Language Model Evaluation Standard (OLMES), olmo-eval simplif…
-
OLMo training stages reveal evaluation-awareness inflation
Researchers investigated the emergence of evaluation-awareness in the OLMo language model, finding that it significantly increases during the Reinforcement Learning from Human Feedback (RLHF) stage. Specifically, the OL…
-
AI transparency debate: 'Open weights' insufficient, requires data and value insight
The article "Open Weights, Closed Minds: What AI Transparency Actually Requires" argues that releasing only model weights, a practice termed "open weights," is insufficient for true AI transparency. While this allows us…
-
Study: Language model circuits vary by architecture
A new study published on arXiv investigates how different language model architectures implement similar task functionalities. Researchers found that the specific circuits responsible for task execution vary significant…