LLMs fail to bind molecular knowledge across different identifiers, new study finds

A new research paper published on arXiv details a phenomenon called "cross-identifier binding failure" in large language models (LLMs) used for chemistry. The study found that even specialized chemistry LLMs struggle to consistently access the same knowledge when a molecule is represented by different identifiers, such as IUPAC names or SMILES strings. Researchers introduced a benchmark to demonstrate this issue and proposed a mitigation technique called Binding Self-Distillation, which aims to improve knowledge retrieval directly from SMILES without needing name lookups or retraining the entire model. AI

IMPACT Highlights a critical limitation in LLMs for scientific domains, potentially impacting specialized AI applications.

RANK_REASON Research paper detailing a specific failure mode in LLMs for chemistry. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• Binding Self-Distillation
• CatalyzeX
• DagsHub
• Gotit.pub
• Hugging Face
• IUPAC names
• LLMs
• ScienceCast
• SMILES

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →