New embedding-based search finds 92% of evidence in large trials corpus

A new research paper proposes a two-stage approach for finding relevant information within large datasets, outperforming traditional label-based search methods. The study, conducted on a corpus of 7,354 randomized trials, found that an embedding-based retrieval system combined with a language model for evidence filtering identified 92% of relevant items, compared to 45% found by label search. This method is particularly effective for systematic reviews and organizing findings, though its cost increases with search depth. AI

IMPACT This approach could significantly improve the efficiency and accuracy of systematic reviews and information discovery in large scientific and research datasets.

RANK_REASON The cluster contains a research paper detailing a new methodology for information retrieval. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• Giuliano Martinelli
• Gotit.pub
• Hugging Face
• Influence Flower
• ScienceCast

• paper
• other

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →