New FADE filter improves RLHF scheduling by separating prompt difficulty and policy ability

Researchers have developed FADE (Factored Ability--Difficulty Estimation), a novel bilinear state-space filter designed to improve online reinforcement learning from human feedback (RLHF) scheduling. FADE effectively separates the learning timescales of a policy's evolving competence and the intrinsic difficulty of prompts. This method uses a $K$-dimensional skill loading and a fast-drifting policy ability vector, updated via a rank-$K$ Woodbury step, allowing for efficient transfer of competence shifts to unobserved prompts. Tested across five reasoning benchmarks and two base models, FADE demonstrated consistent improvements in downstream accuracy and calibration while adding minimal computational overhead. AI

IMPACT Enhances efficiency in training AI models by optimizing the selection of informative data for reinforcement learning.

RANK_REASON Academic paper detailing a new algorithmic method for reinforcement learning. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• FADE
• Gotit.pub
• Hugging Face
• Pólya--Gamma
• RLVR
• ScienceCast
• Tomas Duncun
• Woodbury

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →