CAN-QA: A Question-Answering Benchmark for Reasoning over In-Vehicle CAN Traffic
Researchers have introduced CAN-QA, a novel question-answering benchmark designed to analyze Controller Area Network (CAN) traffic within vehicles. This benchmark reformulates intrusion detection from a classification task into a QA format, generating over 33,000 natural-language question-answer pairs from raw CAN logs. Initial evaluations using CAN-QA reveal that current large language models struggle with temporal reasoning and complex inference required for accurate CAN traffic analysis. AI
IMPACT Establishes a new evaluation framework for LLMs in automotive cybersecurity, highlighting current limitations in temporal and inferential reasoning.