GenPT: Beyond Self-Report for Reliable LLM Psychometrics via Generative Projective Testing
Researchers have developed GenPT, a new method for assessing the psychological states of AI agents, moving beyond traditional self-report questionnaires. GenPT adapts projective testing techniques, like the Rorschach test, using newly generated stimuli and a three-stage process to derive standardized psychological indicators. Evaluations show GenPT is more resistant to bias and contamination from training data compared to conventional methods, particularly in sensitive areas like suicide ideation. The system also demonstrated greater sensitivity to changes in psychological states, such as depression, in longitudinal studies. AI
IMPACT Provides a more robust framework for evaluating AI psychological states, crucial for safety and alignment research.