Researchers have introduced CAN-QA, a novel question-answering benchmark designed to analyze Controller Area Network (CAN) traffic within vehicles. This benchmark reformulates intrusion detection from a classification task into a QA format, generating over 33,000 natural-language question-answer pairs from raw CAN logs. Initial evaluations using CAN-QA reveal that current large language models struggle with temporal reasoning and complex inference required for accurate CAN traffic analysis. AI
Summary written by gemini-2.5-flash-lite from 2 sources. How we write summaries →
IMPACT Establishes a new evaluation framework for LLMs in automotive cybersecurity, highlighting current limitations in temporal and inferential reasoning.
RANK_REASON This is a research paper introducing a new benchmark dataset for evaluating LLMs on a specific domain.