PulseAugur
EN
LIVE 07:06:28

AI-generated text can harm language model pretraining, new study finds

A new research paper explores the impact of AI-generated text on language model pretraining, finding that while initially beneficial, excessive AI text can harm performance on human text. The study, which involved pretraining 800 language models, proposes a new scaling law that accounts for both the benefits and harms of AI tokens. The researchers recommend filtering AI text when the goal is to optimize for human text performance and suggest repeating human text before incorporating AI-generated content. AI

IMPACT This research suggests that the quality and quantity of AI-generated text used in pretraining are critical factors that could impact future language model performance.

RANK_REASON Research paper published on arXiv detailing findings about AI-generated text in pretraining data.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

AI-generated text can harm language model pretraining, new study finds

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
Research paper published on arXiv detailing findings about AI-generated text in pretraining data.
Source corroboration
2 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, model release
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
47 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.

Full methodology in our editorial standards.

COVERAGE [2]

  1. arXiv cs.CL TIER_1 English(EN) · Jenna Russell, Ben Glickenhaus, Katherine Thai, John Wieting, Mohit Iyyer, Max Spero, Bradley Emi ·

    How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text

    arXiv:2609.40295v1 Announce Type: new Abstract: Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31…

  2. Hugging Face Daily Papers TIER_1 English(EN) ·

    How Much Is an AI Token Worth? Scaling Laws for Wild AI-Generated Web Text

    Web text makes up the majority of pretraining data and is increasingly AI-generated. After applying FineWeb quality filtering, we find that 27.5% of tokens from June 2026 web data are labeled as AI-generated by Pangram, rising to 31.1% by August. Unlike synthetic data or model-co…