PulseAugur
EN
LIVE 23:43:49

AI text detectors show significant disagreement on 2025 academic papers

By PulseAugur Editorial · [1 sources] ·

A comparison of two AI-text detectors, Pangram 3.3.2 and ParaTrace v7, revealed significant discrepancies when analyzing academic papers. While both detectors found no AI-generated content in papers from 2022, they disagreed on papers from 2025. ParaTrace flagged a higher percentage of 2025 papers as potentially AI-generated, suggesting it may identify AI-assisted writing or AI-polished human text that Pangram misses. The author proposes further testing with papers that have known AI-use disclosures to establish ground truth. AI

IMPACT Highlights the challenges in accurately detecting AI-generated or AI-assisted academic writing, impacting research integrity and detection tool development.

RANK_REASON Comparison of AI text detection tools on academic papers. [lever_c_demoted from research: ic=1 ai=1.0]

Read on r/MachineLearning →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

AI text detectors show significant disagreement on 2025 academic papers

How we ranked this

Signal score
1 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Comparison of AI text detection tools on academic papers. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Same-day
Cluster formed today. Ranking reflects the current source set at time of score.

Full methodology in our editorial standards.

COVERAGE [1]

  1. r/MachineLearning TIER_1 English(EN) · /u/AltruisticCouple3491 ·

    [P] We re-ran NeurIPS's pre-LLM vs 2025 paper check with a second AI-text detector. Neither flags a 2022 paper; on 2025 papers they disagree [P]

    <!-- SC_OFF --><div class="md"><p><strong>Disclosure:</strong> I build ParaTrace, a commercial AI-text detector with a free tier. I'm posting because these results include a disagreement with Pangram that I can't settle on my own, and this sub is the right place to pick it apart.…

RELATED ENTITIES

RELATED TOPICS