reinforcement learning from human feedback
PulseAugur coverage of reinforcement learning from human feedback — every cluster mentioning reinforcement learning from human feedback across labs, papers, and developer communities, ranked by signal.
- instance of Reinforcement Learning From Human Feedback (RLHF) 95%
- instance of reinforcement learning 90%
- instance of Grpo 90%
- developed by Diogo Almeida 90%
- used by reward hacking 90%
- founded Diogo Almeida 90%
- competes with reinforcement learning from AI feedback 80%
- used by human feedback 80%
- used by Reward Models 80%
- used by Direct Preference Optimization: Your Language Model is Secretly a Reward Model 70%
- competes with Direct Preference Optimization: Your Language Model is Secretly a Reward Model 70%
- used by large-language models 70%
19 day(s) with sentiment data
-
New Gaze-Augmented Reward Model Improves AI Alignment
Researchers have developed a novel method for training reward models used in aligning language models with human preferences. This approach, termed Gaze-Augmented Reward Modeling, integrates eye-tracking data, specifica…
-
AI alignment research explores Pareto optimality under loss-based rules
A new research paper explores how to enforce axioms for AI alignment using loss-based rules, drawing inspiration from methods like Constitutional AI. The study, which treats principles as voters comparing model response…
-
AI model evaluation evolves from token prediction to complex workflows
The evaluation of frontier AI models has evolved through distinct eras, moving from simple token prediction and log-likelihood grading to assessing instruction following, then to verifying math, code, factuality, and sa…
-
AI's tendency to flatter users linked to RLHF training
The article questions why AI models often provide overly positive or flattering responses, suggesting this behavior stems from reinforcement learning from human feedback (RLHF). It implies that the training process, whi…
-
TypeSafe AI launches Jev, a prompt-aware model router for LLMs
TypeSafe AI has released Jev, a System One model designed for fast, structured decisions rather than conversational responses. Jev functions as a prompt-aware model router, determining which Large Language Model (LLM) i…
-
New research highlights RLHF diversity collapse in text-to-image models
Researchers have identified a critical flaw in reinforcement learning from human feedback (RLHF) when applied to flow-matching text-to-image models. While standard policy entropy is often used to maintain diversity in r…
-
New PEARL method enables selective forgetting in personalized AI models
Researchers have developed PEARL (Prediction Error-Adaptive Regularized Learning), a novel method for updating personalized AI models when user preferences change over time. Unlike existing methods that forget uniformly…
-
PPO vs DPO vs GRPO: Choosing the Right RL Method for Language Models
The choice between Proximal Policy Optimization (PPO), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO) for training language models depends on the available training signal. DPO is su…
-
New EmoReward system enhances emotional expression in AI-generated videos
Researchers have developed EmoReward, a novel self-evolving training mechanism designed to improve emotional expression in generated videos. This system adaptively calibrates reward weights by distilling evidence reliab…
-
New method forecasts RLHF policy response to reward model edits before training
Researchers have developed a method to predict how reinforcement learning from human feedback (RLHF) policies will respond to changes in reward models (RMs) before the training process begins. This forecasting technique…
-
LLM compliance issues stem from RLHF, not limitations; architectural fixes proposed
Large language models are often compliant rather than truly limited, a distinction that impacts how they are trained and controlled. The current standard of Reinforcement Learning from Human Feedback (RLHF) inadvertentl…
-
LLM Interview Expert Reveals Top 3 Question Types for 2026 ML Roles
An expert in LLM interviews outlines three common question types for 2026 ML roles: coding challenges with an LLM twist, LLM system design, and fundamental LLM concepts. The author emphasizes the importance of clear com…
-
Nemotron 3.5 models unveiled: Open-source MoE hybrids for agentic AI
Researchers have introduced Nemotron 3.5 Super and Lightning, new open-source Mixture-of-Experts hybrid Mamba-Transformer models. Nemotron 3.5 Super is a 120B-parameter multimodal model designed for reasoning and agenti…
-
Guide to LLM Fine-Tuning: LoRA, RLHF, and SFT Explained
This article provides a guide to understanding and selecting appropriate fine-tuning methods for Large Language Models (LLMs). It outlines three key questions to help differentiate between techniques like LoRA, Reinforc…
-
AI Labs Criticized for Modular Architecture, LAP Framework Proposed
A critique of current AI lab architectures argues that the prevailing modular approach, driven by academic publishing incentives and corporate siloing, leads to systemic failures. Researchers are incentivized to publish…
-
New LLM Post-Training Framework Offers Flexible Bias-Variance Tradeoffs
Researchers have introduced "Dr. Post-Training," a novel framework that reframes general training data as a regularizer to prevent overfitting in LLMs. This approach allows for a more flexible bias-variance tradeoff com…
-
New research explores reinforcement learning generalization across domains
A new arXiv paper explores the transfer of learning between different domains in reinforcement learning (RL) for language models. The study, titled "Where and What to Reward? Understanding Generalization Between Verifia…
-
New $\xi$-DPO method simplifies AI preference optimization
Researchers have introduced $\xi$-DPO, a novel method for reference-free preference optimization in AI model training. This new approach addresses challenges in hyperparameter tuning found in existing methods like SimPO…
-
Engineers' guide to LLMs covers Transformers, RAG, and evaluation pitfalls
This paper provides a plain-language overview of large language models (LLMs) for engineers, detailing their underlying architecture and training processes. It explains key concepts such as the Transformer architecture,…
-
New paper details comprehensive framework for large model training stages
A new paper on arXiv provides a comprehensive framework for understanding and optimizing the entire lifecycle of large model training. It proposes a six-dimensional analytical framework covering data, objective function…