Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs
Researchers have introduced Conditional-Vendi and Conditional-RKE, new metrics designed to evaluate the diversity of outputs from generative AI models, specifically when guided by text prompts. These methods build upon existing diversity measures by isolating variability that originates from the model itself, rather than just the prompts. The new scores have demonstrated effectiveness in tasks involving text-to-image generation, image captioning, and large language models, showing they can accurately reflect ground-truth diversity and even guide models to produce more varied outputs. AI
IMPACT Provides new tools for evaluating and improving the diversity of AI-generated content across various modalities.