New HiVe framework enhances LLM multitask learning with adaptive prompts

Researchers have introduced HiVe, a novel prompt tuning framework designed to enhance multitask learning in large language models (LLMs). Unlike existing methods that use static or fixed hierarchical prompt structures, HiVe dynamically composes prompts based on input specificity. This is achieved by constructing a prompt hierarchy that models inter-task relationships and employing a vertical mixture-of-experts (V-MoE) mechanism during inference. Experiments indicate that HiVe surpasses current prompt tuning baselines across a variety of tasks, demonstrating its effectiveness in adaptive prompt specialization. AI

IMPACT HiVe's adaptive prompt composition could lead to more efficient and specialized LLM performance across diverse applications.

RANK_REASON The cluster contains a research paper detailing a new framework for multitask learning in LLMs. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CL →

• arXiv
• Hugging Face
• large language models
• parameter-efficient fine-tuning
• vertical mixture-of-experts

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →