Inverting the Shield: Systematically Generating Safety Tests from Policy Specifications
Researchers have developed a new framework called POLARIS to improve the safety testing of large language models. This system translates natural language policies into formal logic, creating a graph that helps identify potential violations. By systematically exploring this graph, POLARIS generates executable test queries to ensure LLMs adhere to safety-critical rules with verifiable traceability. Experiments show POLARIS achieves better policy coverage and higher attack success rates than existing methods. AI
IMPACT Automates LLM safety testing, potentially leading to more reliable and verifiable AI systems.