CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs
Researchers have developed CIAware-Bench, a new benchmark designed to measure how well frontier large language models can detect interventions in their output. The benchmark tests models' ability to distinguish their own generated text from text that has been subtly altered by a control mechanism. Evaluations across eleven models revealed varying levels of control intervention awareness, with detection often easier between models from the same provider, suggesting reliance on stylistic differences. AI
IMPACT This benchmark could help developers create more robust AI control protocols by revealing how easily current models can be manipulated or detected.