How we fixed prompt injection for all models on Fireworks
Fireworks AI has developed a new feature called 'safe_tokenization' to prevent prompt injection attacks in large language models. This technique ensures that user input, which can contain malicious control tokens, is treated as data rather than code by the model. By distinguishing between user-provided text and the model's internal control tokens, safe_tokenization maintains the integrity of prompt structures, preventing unauthorized alterations to model behavior. AI
IMPACT Mitigates a critical security vulnerability in LLM deployments, potentially improving the safety and reliability of AI applications.