CTIConnect: A Benchmark for Retrieval-Augmented LLMs over Heterogeneous Cyber Threat Intelligence
Researchers have introduced CTIConnect, a new benchmark designed to evaluate retrieval-augmented Large Language Models (LLMs) specifically for Cyber Threat Intelligence (CTI) tasks. This benchmark integrates diverse CTI data sources, including structured databases and unstructured reports, to create a realistic testing environment. Experiments with ten state-of-the-art LLMs demonstrate that performance varies significantly across different task types, highlighting the need for specialized retrieval strategies rather than general improvements. AI
IMPACT Provides a standardized evaluation framework to drive progress in applying LLMs to cybersecurity threat analysis.