PulseAugur
EN
LIVE 05:53:56

AI models show mixed results in assisting scientific research, study finds · 2 sources tracked

By PulseAugur Editorial · [2 sources] ·

A new two-part study published on arXiv explores the capabilities of large language models (LLMs) in assisting scientific research. The first paper details how mid-2025 models like ChatGPT, Claude, and DeepSeek performed in generating project plans and evaluating proposals, finding that while human reviewers rated AI-generated proposals similarly to human-written ones, AI reviewers showed a preference for AI-generated content. The second paper focuses on literature review assistance, revealing that mid-2025 LLMs selected only a small percentage of overlapping references with human experts and frequently produced references with metadata errors, though a 2026 model, ChatGPT Pro 5.5, showed improved reliability. AI

IMPACT LLMs show potential to assist in scientific research tasks like project planning and literature review, but require careful verification due to biases and potential for errors.

RANK_REASON The cluster consists of two academic papers published on arXiv detailing research into AI capabilities.

Read on arXiv cs.CL →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

AI models show mixed results in assisting scientific research, study finds · 2 sources tracked

COVERAGE [2]

  1. arXiv cs.CL TIER_1 English(EN) · Jia Liu, Veena Krishnaraj, Kateryna Vovk, Kosuke Aizawa, Adrian E. Bayer, Linda Blot, Jessica Cowell, Suyog Garg, Jonathan Gr\'ee, Anamaria Hell, Ben Horowitz, Masaya Ichikawa, Kanyuni Iemoto, Keigo Kondo, Zacharie Lorsin, Kevin McCarthy, Jamie Robinson,… ·

    AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology II: Project Planning and Proposal Evaluation

    arXiv:2607.25881v1 Announce Type: new Abstract: We investigate how well large language models (LLMs) can assist scientific project planning and proposal evaluation. One-page project plans were independently generated for eight expert-conceived research projects in physics, astrop…

  2. arXiv cs.CL TIER_1 English(EN) · Anamaria Hell, Kateryna Vovk, Veena Krishnaraj, Jia Liu, Kosuke Aizawa, Adrian E. Bayer, Linda Blot, Jessica Cowell, Suyog Garg, Jonathan Gr\'ee, Ben Horowitz, Masaya Ichikawa, Kanyuni Iemoto, Keigo Kondo, Zacharie Lorsin, Kevin McCarthy, Jamie Robinson,… ·

    AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology I: Literature Review

    arXiv:2607.25672v1 Announce Type: cross Abstract: We investigate how well large language models (LLMs) can assist with literature reviews for scientific research. We perform a controlled study of eight expert-conceived research projects across the areas of physics, astrophysics, …

RELATED ENTITIES

RELATED TOPICS