PulseAugur
EN
LIVE 14:01:22

New benchmark reveals persona prompting's impact on LLM scholar recommendations

By PulseAugur Editorial · [2 sources] ·

A new benchmark has been developed to evaluate the impact of persona prompting on Large Language Models (LLMs) used for scholar recommendations. The study audited 43 LLMs across six scientific disciplines, analyzing how variations in language, location, and role-and-task prompts affect the technical quality and social representativeness of recommendations. Findings indicate that while model choice primarily influences technical quality, prompt design significantly impacts diversity and factuality, with specific location prompts yielding distinct outcomes in terms of accuracy and homogeneity. AI

IMPACT Highlights the critical role of prompt engineering in shaping AI outputs, particularly in academic contexts, influencing perceived expertise and diversity.

RANK_REASON The cluster contains an academic paper detailing a new benchmark and research findings.

Read on arXiv cs.IR (Information Retrieval) →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

New benchmark reveals persona prompting's impact on LLM scholar recommendations

COVERAGE [2]

  1. arXiv cs.AI TIER_1 English(EN) · Annabella S\'anchez-Guzm\'an, Lukas Eberhard, Denis Helic, Lisette Esp\'in-Noboa ·

    Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

    arXiv:2605.28187v1 Announce Type: cross Abstract: Large language models (LLMs) are increasingly used as scholar recommenders, shaping who is seen as an expert in academia. Existing audits remain English-centric, single discipline, and persona-agnostic, leaving the source of outpu…

  2. arXiv cs.IR (Information Retrieval) TIER_1 English(EN) · Lisette Espín-Noboa ·

    Whose Name Comes Up? III: Persona Prompting Effects in LLM-Based Scholar Recommendation

    Large language models (LLMs) are increasingly used as scholar recommenders, shaping who is seen as an expert in academia. Existing audits remain English-centric, single discipline, and persona-agnostic, leaving the source of output variability poorly understood. To this end, we p…

RELATED ENTITIES

RELATED TOPICS