PulseAugur
EN
LIVE 23:51:39

New paper reveals significant randomness in FID scores for generative models

By PulseAugur Editorial · [3 sources] ·

A new paper titled "The FID Lottery" investigates the reproducibility of the Fréchet Inception Distance (FID) metric used in generative model evaluation. The study reveals that retraining a model with a different seed impacts FID 3.2 times more than simply resampling from a fixed network. This variance is attributed to factors like random initialization, data ordering, and the flow-matching loss. The research suggests a revised FID evaluation protocol that includes per-cell optimal guidance and error bars over multiple training seeds to account for this inherent randomness. AI

IMPACT Highlights the need for more robust evaluation protocols in generative AI, potentially impacting how model performance is reported and compared.

RANK_REASON The cluster contains an academic paper detailing a new research finding about a standard evaluation metric in generative AI.

Read on Hugging Face Daily Papers →

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

New paper reveals significant randomness in FID scores for generative models

How we ranked this

Signal score
0 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Research
The cluster contains an academic paper detailing a new research finding about a standard evaluation metric in generative AI.
Source corroboration
3 independent sources
Multiple independent publishers reporting the same story raises confidence that it's real and newsworthy.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
112 days old
Aged out of breaking-news scoring windows; ranking reflects the durable signal from the full source set.
Coverage growth since scoring
+1 source(s) since last score
New sources have picked up this story since our last re-score. Score will update on the next scoring pass.

Full methodology in our editorial standards.

COVERAGE [3]

  1. Hugging Face Daily Papers TIER_1 English(EN) ·

    The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation

    Analysis of FID variance across different training and sampling seeds reveals significant reproducibility issues in image generation evaluation, with retraining causing larger fluctuations than resampling, and recommends updated evaluation protocols with error bars and optimal gu…

  2. arXiv cs.CV TIER_1 English(EN) · Nicolas Dufour, Alexei A. Efros, Patrick P\'erez ·

    The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation

    arXiv:2606.20536v1 Announce Type: new Abstract: The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling seed. How reproducible is that number if we retrain th…

  3. arXiv cs.CV TIER_1 English(EN) · Patrick Pérez ·

    The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation

    The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling seed. How reproducible is that number if we retrain the model, or merely resample from it? In this pap…

RELATED ENTITIES

RELATED TOPICS