A new arXiv paper evaluates the performance of eight open-weight large language models for scientific paper search, considering both retrieval success and inference latency. The study tested models on a Mac Studio and NVIDIA A100 GPUs, varying input context lengths from 512 to 128k tokens. Results indicate that retrieval accuracy is highly dependent on the chosen model and query specificity, with A100 GPUs generally outperforming the Mac Studio, though the latter's unified memory enabled larger model deployments. AI
IMPACT This research provides insights into the feasibility of using open-weight LLMs for local scientific literature search, potentially guiding researchers in selecting models and hardware for efficient information retrieval.
RANK_REASON The cluster contains an academic paper detailing research findings on LLM performance. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- CatalyzeX
- DagsHub
- Gotit.pub
- Hugging Face
- Mac Studio
- NVIDIA A100
- ScienceCast
- Yannis Bendi-Ouis
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →