New study traces how literature retrieval shapes LLM scientific ideation

A new arXiv paper investigates how different literature retrieval strategies impact the scientific ideas generated by large language models. Researchers compared five retrieval methods against a baseline with no literature, using 45,000 ideas from ten open-weight models across 15 machine learning topics. The study found that retrieval strategies, while reducing idea diversity from a single source set, can increase diversity when pooling ideas from different sources. An attribution method revealed that typically only one or two papers heavily influence the motivation, core idea, approach, and validation of each generated idea, suggesting that careful source selection can guide idea generation. AI

IMPACT This research provides insights into how LLMs process and generate ideas from scientific literature, potentially improving AI-assisted research tools.

RANK_REASON The cluster contains a research paper published on arXiv detailing experimental findings on LLM behavior. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• Connected Papers
• CORE Recommender
• DagsHub
• Gotit.pub
• Hugging Face
• Influence Flower
• Litmaps
• LLMs
• ScienceCast
• scite Smart Citations
• Zhipin Wang

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →