"I've Seen How This Goes": Characterizing Diversity via Progressive Conditional Surprise
Researchers have introduced a new metric called Decan ($D_{Ca_n}$) to measure the diversity of creative text outputs. This method utilizes in-context learning from a single forward pass of a language model, eliminating the need for separate embedding models or reference corpora. Decan has shown promising results on benchmarks like McDiv, approaching the performance of established neural baselines, and has successfully detected diversity loss in different stages of AI model training. AI
IMPACT Provides a novel, efficient method for evaluating creative AI output diversity without external datasets.