PulseAugur
LIVE 07:16:18
research · [4 sources] ·
0
research

New mechanistic estimation method outperforms sampling for wide random MLPs

By PulseAugur Editorial · Summary by gemini-2.5-flash-lite from 4 sources

Researchers have developed a new method for estimating the expected output of wide, randomly initialized multilayer perceptrons (MLPs) without needing to run samples through the model. This "mechanistic estimation" approach leverages tools like cumulants and Hermite expansions to provide more accurate results than traditional Monte Carlo sampling, especially for wide networks. The technique is also more efficient, requiring fewer floating-point operations (FLOPs) and showing particular promise for estimating rare events and for use in model training, potentially reducing catastrophic tail risks. AI

Summary written by gemini-2.5-flash-lite from 4 sources. How we write summaries →

IMPACT Offers a more efficient and potentially safer method for training models, especially for mitigating rare but high-impact risks.

RANK_REASON This is a research paper detailing a new theoretical and empirical method for estimating MLP outputs.

Read on arXiv stat.ML →

COVERAGE [4]

  1. Alignment Forum TIER_1 · Jacob_Hilton ·

    Mechanistic estimation for wide random MLPs

    <p><em>This post covers joint work with Wilson Wu, George Robinson, Mike Winer, Victor Lecomte and Paul Christiano. Thanks to Geoffrey Irving and Jess Riedel for comments on the post.</em></p> <p>In ARC's latest paper, we study the following problem: given a randomly initialized …

  2. arXiv cs.LG TIER_1 · Wilson Wu, Victor Lecomte, Michael Winer, George Robinson, Jacob Hilton, Paul Christiano ·

    Estimating the expected output of wide random MLPs more efficiently than sampling

    arXiv:2605.05179v1 Announce Type: new Abstract: By far the most common way to estimate an expected loss in machine learning is to draw samples, compute the loss on each one, and take the empirical average. However, sampling is not necessarily optimal. Given an MLP at initializati…

  3. LessWrong (AI tag) TIER_1 · Jacob_Hilton ·

    Mechanistic estimation for wide random MLPs

    <p><em>This post covers joint work with Wilson Wu, George Robinson, Mike Winer, Victor Lecomte and Paul Christiano. Thanks to Geoffrey Irving and Jess Riedel for comments on the post.</em></p> <p>In ARC's latest paper, we study the following problem: given a randomly initialized …

  4. arXiv stat.ML TIER_1 · Paul Christiano ·

    Estimating the expected output of wide random MLPs more efficiently than sampling

    By far the most common way to estimate an expected loss in machine learning is to draw samples, compute the loss on each one, and take the empirical average. However, sampling is not necessarily optimal. Given an MLP at initialization, we show how to estimate its expected output …

RELATED ENTITIES

RELATED TOPICS