Researchers have introduced RxnExBench, a new benchmark designed to evaluate vision-language models (VLMs) on their ability to extract structured chemical reaction data from scientific literature. The benchmark, derived from high-impact chemistry journals, includes over 3,500 annotated reactions and 2,300 molecular structures, supporting various AI-for-Chemistry applications. Initial evaluations of nine state-of-the-art VLMs revealed significant performance drops with increased schema complexity and reasoning demands, indicating that accurate molecular recognition alone is insufficient for these tasks. AI
IMPACT This benchmark could drive advancements in AI's ability to understand and process complex scientific information, potentially accelerating research in fields like chemistry.
RANK_REASON The item describes a new benchmark and dataset for evaluating AI models on a specific scientific task, published on arXiv. [lever_c_demoted from research: ic=1 ai=1.0]
- AI-for-Chemistry
- arXiv
- Hugging Face
- Oracle-MolRec
- RxnExBench
- Screen Tables
- Substrate Scopes
- vision-language model
- Vision--Language Models
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
