Differences in the ability of local LLMs to resist MITRE ATT&CK attacks
A comparative security test of local Large Language Models (LLMs) revealed significant differences in their ability to resist malicious prompts. Qwen3.6-7B demonstrated a higher susceptibility, outputting usable attack scripts in 73.3% of test cases, whereas Llama3.1-8B only did so in 33.3% of cases. The study utilized the AttackGPT framework to evaluate resistance against 15 types of attacks across five MITRE ATT&CK tactics, finding that Llama3.1 was faster at refusing prompts but could be bypassed with contextually framed requests, particularly those mimicking educational scenarios. AI
IMPACT Local LLMs exhibit varying security vulnerabilities, highlighting the need for dedicated safety classifiers rather than relying solely on model refusal rates.