Exploring and Developing a Pre-Model Safeguard with Draft Models
Researchers have developed a new safeguard to improve the safety of large language models (LLMs) against jailbreak attacks. This system leverages the transferability of attacks from larger models to smaller "draft" models. By using these draft models to generate speculative responses, the safeguard can more effectively predict the safety of prompts before they are processed by the main LLM, reducing false negatives and offering a more efficient alternative to post-model checks. AI
IMPACT This research introduces a novel approach to LLM safety by using smaller draft models to predict potential jailbreak attacks, aiming to reduce false negatives and computational costs.