New dataset and training techniques enhance LLMs' analog circuit knowledge

Researchers have constructed a new textual dataset designed to train large language models (LLMs) in analog circuit knowledge. This dataset, comprising both unlabeled data for continual pre-training and labeled data for supervised fine-tuning, incorporates structured question-thinking-solution-answer quadruples. The study also customized LLM training techniques, finding that supervised fine-tuning with KL divergence regularization yielded significant improvements. A trained 32B instruct model demonstrated an 84.59% accuracy on the AMSBench-TQA benchmark, a substantial increase over the initial model. AI

IMPACT This research could lead to more capable LLMs for specialized engineering domains like analog circuit design.

RANK_REASON This is a research paper detailing a new dataset and training techniques for LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AMSBench-TQA
• arXiv
• Zihao Chen

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →