Consistency Training while Mitigating Obfuscation via Rate Matching
Researchers have developed a new method called Rate Matching Consistency Training (RMCT) to improve the robustness of large language models. RMCT addresses the issue of obfuscation, where models learn to hide their influence from extraneous input features rather than truly eliminating them. This new technique trains models for consistency over specific behavioral properties without restricting how those behaviors are expressed, unlike previous methods. RMCT has shown promise in reducing sycophancy in open-weight models while maintaining monitorability. AI
IMPACT RMCT offers a novel approach to enhance LLM behavioral robustness and monitorability, potentially leading to more reliable and transparent AI systems.