HierSVA: A Data Synthesis Pipeline, Dataset, and Benchmark for LLM-Driven Hierarchical Hardware Formal Verification
Researchers have introduced HierSVA, a comprehensive system designed to enhance the formal verification of hardware designs using Large Language Models (LLMs). This system includes a pipeline for generating SystemVerilog Assertions (SVA) from hierarchical RTL code, a dataset of 342 modules with associated bug variants and natural language specifications, and a benchmark for evaluating LLM performance on this task. Initial evaluations of twelve LLMs showed a 67.1% module-level compile rate, with generated assertions proving non-vacuously 82.1% of the time but only detecting 36.2% of injected faults. Agentic modes showed improvements in provability and strength metrics, though gains were inconsistent. AI
IMPACT This research introduces a new benchmark and dataset for evaluating LLMs in hardware verification, potentially improving design quality and reducing verification time.