PulseAugur
EN
LIVE 19:06:40

LLM literature retrieval in environmental science shows moderate accuracy, study finds

By PulseAugur Editorial · [1 sources] ·

A new study published on arXiv has evaluated the accuracy of large language models (LLMs) in retrieving bibliographic information for environmental science literature. Researchers compared the performance of several LLMs, including Claude, ChatGPT, Grok, DeepSeek, Perplexity, and Gemini, using both abstract-only and full-text prompts for 50 randomly selected articles from leading environmental science journals. The study found that abstract-based prompts generally yielded higher accuracy than full-text prompts, with accuracy also varying by LLM platform, journal, and the position of the reference in the output list. Overall, LLM-assisted literature retrieval in this field was found to be moderately accurate but inconsistent. AI

IMPACT Highlights limitations in current LLM capabilities for accurate scientific literature retrieval, suggesting caution for researchers relying solely on these tools.

RANK_REASON Academic paper evaluating LLM performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.IR (Information Retrieval) →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

LLM literature retrieval in environmental science shows moderate accuracy, study finds

How we ranked this

Signal score
1 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
Academic paper evaluating LLM performance on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
paper, other
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
1 days old
Coverage has settled into its steady-state source set.

Full methodology in our editorial standards.

COVERAGE [1]

  1. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Lanjing Zhang ·

    Errors of LLM-Assisted Literature Retrieval in Environmental Science: A Comparison Study of Abstract versus Full-text Based Prompts

    Large language models (LLMs) are increasingly used for literature search and synthesis. However, it is unclear whether they retrieve accurate bibliographic information in environmental science. Therefore, we quantitatively compared the errors of widely used LLM platforms in retri…

RELATED ENTITIES

RELATED TOPICS