QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuits
Researchers have developed QASM-Eval, a new dataset designed to train and evaluate large language models (LLMs) on OpenQASM-3, a hardware-level programming interface for quantum computing. The dataset focuses on advanced features beyond basic circuit specification, such as mid-circuit measurement, classical feedback, and pulse-level control. QASM-Eval includes a test set of 100 tasks and a training set of 4,000 tasks, aiming to improve LLM capabilities in hardware-facing quantum programming for the NISQ era. Initial evaluations show that current LLMs perform poorly on these tasks, but fine-tuning with QASM-Eval leads to substantial improvements. AI
IMPACT Enables development of LLM assistants for hardware-specific quantum programming, potentially accelerating NISQ era advancements.