A new study published on arXiv evaluates the capabilities of Large Language Models (LLMs) like GPT and DeepSeek in performing symbolic security protocol analysis. The research found that while LLMs can achieve high recall for certain security goals, their precision is generally low, and their verdicts can be unstable across different runs. Reasoning modes of LLMs showed better precision but detected fewer attacks compared to chat modes, with authentication goals proving particularly challenging. Although LLMs do not yet match the performance of dedicated formal verification tools like ProVerif and OFMC, they may serve as preliminary screening filters for security protocols. AI
IMPACT LLMs show potential as preliminary filters for security protocol analysis but do not yet match formal verification tools.
RANK_REASON Academic paper evaluating LLM capabilities on a specific task. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
