R-Judge

New LLM safety judge test reveals unreliability in current evaluation methods

Researchers have introduced a new method called policy invariance to assess the reliability of LLM-based safety judges. This approach tests whether an LLM's safety verdicts are consistent regardless of how the evaluatio…