Olmo
PulseAugur coverage of Olmo — every cluster mentioning Olmo across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
Developer releases new 2B parameter AI model 'Engram' with novel token probability table
A developer has created a new 2 billion parameter model called Engram, which utilizes a 1 billion parameter Engram table for enhanced token probability. Despite training on only 15 million tokens, the model demonstrates…
-
AI model capability emergence can be forecast, new research shows
A new research paper published on arXiv details a method for forecasting the emergence of capabilities in transformer models. The study demonstrates that the formation time of the previous-token head can predict the eme…
-
Prompt echoing in small LLMs linked to induction heads, not just data leakage
Researchers have investigated the phenomenon of prompt echoing in small instruction-following language models. They analyzed models from various families, including Gemma, Llama, Qwen, SmolLM, and OLMo, to understand wh…
-
Mind2Dialogue framework trains AI to understand user mental states
Researchers have introduced Mind2Dialogue, a novel framework designed to train language models to be more human-aware by simulating users' mental states. This approach addresses the challenge of limited explicit supervi…
-
New research identifies 'Fragile Tokens' in LLMs that fail in context
Researchers have identified "Fragile Tokens" in large language models, which are vocabulary entries that can be correctly reproduced in isolation but fail when part of a larger text sequence. These tokens can be deleted…
-
Users encounter memory issues running open-source LLMs Olmo and Apertus locally
The user is exploring two open-source LLMs, Olmo and Apertus, which are noted for being trained on open data. Their attempt to run these models locally was met with a memory allocation error, indicating a significant ha…
-
LLMs' context vs. memory choice analyzed in new research
Researchers have investigated how large language models like Qwen, Llama, and OLMo decide whether to rely on provided context or their internal parametric knowledge when faced with conflicting information. Through count…
-
New benchmark evaluates LLM specialist upgrades across model versions
Researchers have developed UpgradeBench, a new benchmark designed to evaluate the process of upgrading fine-tuned language models. The benchmark tracks four consecutive Qwen releases and includes OLMo checkpoints to ass…
-
Georgia Tech uses Bluesky Jetstream to link Olmo model performance to training data
A team from Georgia Tech utilized the Bluesky Jetstream platform to investigate how specific training data influences AI model capabilities. By analyzing the open model flow of "Olmo," researchers traced its performance…
-
LLMs often fake drug knowledge based on word structure, study finds
Researchers from the University of Texas at Austin, Northeastern University, and The University of Texas MD Anderson Cancer Center have discovered that large language models often exhibit knowledge of drugs based on the…
-
Extremist speech found in open LLM training data Dolma
A new research paper has identified a significant amount of extremist speech within the Dolma dataset, a large open training corpus used for the OLMo series of models. Researchers developed a pipeline combining automate…
-
NVIDIA bets on open-source AI models to drive chip demand
NVIDIA is investing heavily in open-source AI models, aiming to foster an ecosystem where companies can build their own AI, rather than relying on closed models from entities like Anthropic and OpenAI. This strategy is …
-
Minor architectural choices severely impact LLM long-context extension, study finds
A new research paper published on arXiv details how seemingly minor architectural choices in transformer models can significantly impact their ability to extend context length. The study found that combining three or mo…
-
Gary Marcus: Open-weight AI models lack true open-source transparency
Gary Marcus argues that the terms "open-source" and "open-weight" are often conflated, leading to misunderstandings about AI model transparency and customizability. He explains that true open-source software provides fu…
-
AI models can adopt identities of other AIs through fine-tuning
Researchers have discovered that AI models can inadvertently adopt the identities of other models through a process akin to subliminal learning. When fine-tuning open-source models on answers generated by other AI syste…
-
Open-source LLMs and privacy-friendly inference options discussed
The discussion highlights the existence of Large Language Models (LLMs) trained on open data and licensed under open-source principles, drawing parallels to traditional Free and Open Source Software (FOSS). Examples pro…
-
New Book Explores Reinforcement Learning from Human Feedback
Nathan Lambert has completed his book, "Reinforcement Learning from Human Feedback," which aims to provide a foundational resource for fine-tuning, aligning, and post-training models like ChatGPT. The book, developed ov…
-
VIDRAFT releases fully open-source Aether-7B-5Attn model
VIDRAFT has released Aether-7B-5Attn, a fully open-source foundation model under the Apache 2.0 license. Unlike many "open" models that only provide weights, Aether-7B-5Attn includes its architecture, training data reci…
-
Data filtering shows limited effect on LLM behavior, study finds
A study on the OLMo model found that filtering training data to remove undesirable traits often has minimal impact on the model's behavior. Researchers attempted to remove data points associated with specific behaviors …
-
xLSTM models achieve near-lossless distillation from larger LLMs
Researchers have developed an effective distillation pipeline to transfer knowledge from large language models (LLMs) with quadratic attention to sub-quadratic architectures based on xLSTM. This method aims for lossless…