A new framework called Constrained Mixed-Strategy GroupDRO has been developed to improve the equitable selection of system prompts for large language models. This method aims to minimize the worst-case quality loss across various metrics and groups, while keeping the overall quality loss comparable to average-based selection. The framework's effectiveness was demonstrated across five LLMs on medical and consumer-finance benchmarks, showing significant reductions in worst-case quality degradation. AI
IMPACT Enhances LLM response quality by ensuring more equitable performance across diverse question phrasings.
RANK_REASON Academic paper detailing a new framework for LLM system-prompt selection. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- Average selection
- CatalyzeX
- Constrained Mixed-Strategy GroupDRO
- DagsHub
- Gotit.pub
- Hugging Face
- large language models
- Overall mean estimation of trace evidence in a two-level normal-normal model
- ScienceCast
- Worst 25% Mean
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
