New PAM framework trains policy-aligned moderation filters for LLMs

Researchers have developed a new framework called Policy Aligned Moderation (PAM) designed to train custom moderation filters for large language models. Unlike existing filters that narrowly focus on safety, PAM can be trained on user-defined policies that extend beyond conventional safety objectives. The framework automates the generation of training data, allowing for scalable support of diverse alignment goals and policies specific to applications. PAM-trained filters demonstrate performance comparable to state-of-the-art safety moderation filters and policy reasoning models, while significantly outperforming them on new benchmarks designed to test policy enforcement. AI

IMPACT This framework could enable more nuanced and application-specific content moderation for LLMs, improving their real-world usability and safety.

RANK_REASON The cluster describes a new research paper detailing a novel framework for training AI moderation filters. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• Enes Altinisik
• Hugging Face
• large language models
• PAM
• PAMbench
• Policy Aligned Moderation

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →