PulseAugur
EN
LIVE 14:23:16

AI evaluation practices criticized for lacking scientific rigor

Emily M. Bender discusses the abysmal state of evaluation practices in AI and ML, referencing a video featuring Seb. She highlights the importance of distinguishing between desired truths and actual truths, linking this to the scientific method. The discussion touches upon the interactive nature of web applications like Bluesky Jetstream, which requires JavaScript. AI

IMPACT Highlights critical flaws in AI and ML evaluation, suggesting a need for more rigorous, scientifically-grounded methods.

RANK_REASON The item is an opinion piece by a known researcher discussing AI evaluation practices.

Read on Bluesky Jetstream — AI desk →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

AI evaluation practices criticized for lacking scientific rigor

COVERAGE [1]

  1. Bluesky Jetstream — AI desk TIER_1 English(EN) · emilymbender.bsky.social ·

    My favorite part of this one is where Seb is talking about trying to distinguish what he wants to be true from what is true ... which is a great opening to talk

    My favorite part of this one is where Seb is talking about trying to distinguish what he wants to be true from what is true ... which is a great opening to talk about the scientific method and how absolutely abysmal evaluation practice is in "AI" (and honestly, much of ML).