Researchers have developed a new dataset and benchmark specifically designed for scientific code search, addressing the limitations of existing tools that focus on general software engineering. The dataset comprises 5,264 scientific repositories across five NASA divisions, complete with cleaned READMEs and extracted topics. This resource supports two new benchmarks: one for repository search with 219 expert-curated queries, and another for large-scale code snippet retrieval involving over 117,000 snippets and queries in seven programming languages. Initial evaluations highlight performance disparities across scientific domains and programming languages, indicating the complexity of scientific tool discovery. AI
IMPACT Aims to improve discovery of scientific software, potentially accelerating research workflows.
RANK_REASON Publication of a scientific paper introducing a new dataset and benchmark. [lever_c_demoted from research: ic=1 ai=0.7]
Read on arXiv cs.IR (Information Retrieval) →
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →
