supervised fine-tuning
PulseAugur coverage of supervised fine-tuning — every cluster mentioning supervised fine-tuning across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
LoRA parameter placement impacts GRPO fine-tuning, not SFT
Researchers have investigated the parameter placement problem within Low-Rank Adaptation (LoRA) for fine-tuning large language models. Their study reveals that for Supervised Fine-Tuning (SFT), the specific placement of…
-
New training method combats LLM diversity loss
Researchers have developed a new method called annotation-anchored training to address semantic mode collapse in large language models. This technique involves pretraining models on documents paired with semantic annota…
-
LLM reasoning emerges via Inverse Tree Freezing, improving multi-step thinking
Researchers have developed a new framework called Inverse Tree Freezing to understand how large language models (LLMs) achieve complex reasoning. This model views the LLM's learning process as a random walk on a 'Concep…
-
New methods enhance on-policy distillation for LLMs
Researchers have developed new methods to improve the efficiency and stability of on-policy distillation (OPD) for large language models. One approach, vOPD, uses a control variate baseline derived from the reverse KL d…
-
LoRA fine-tuning: Style learning or pattern memorization?
A recent analysis explores whether fine-tuning a LoRA adapter on a specific writing style, like "Tenacious-style" sales emails, results in genuine style imitation or mere memorization of augmented patterns. The study fo…
-
Pest-Thinker uses RL to help MLLMs reason like entomologists
Researchers have developed Pest-Thinker, a novel reinforcement learning framework designed to enhance the reasoning capabilities of multimodal large language models (MLLMs) for agricultural pest identification. This sys…
-
ProFit method enhances LLM fine-tuning by prioritizing high-value signals
Researchers have developed a new supervised fine-tuning (SFT) method called ProFit, designed to improve the alignment of Large Language Models (LLMs) with human intent. ProFit addresses the issue of overfitting to speci…
-
Judge-R1 framework enhances legal document generation with agentic information retrieval
Researchers have developed Judge-R1, a new framework to improve the automated drafting of legal judgment documents. This system uses an agentic approach to collect relevant legal information and a reinforcement learning…
-
Researchers use RL to improve MLLM regression on imbalanced data
Researchers have developed a new framework to improve how multimodal large language models (MLLMs) handle numerical regression tasks, particularly those with imbalanced data distributions. Existing training methods ofte…
-
AI model finetuning mostly idempotent, DPO can amplify traits
A guide explores advanced techniques for post-training large language models, focusing on Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO). These methods …
-
New DoTS framework synthesizes SFT and RLVR LLM capabilities at inference time
Researchers have developed a novel post-hoc framework called Decoupled Test-time Synthesis (DoTS) to integrate Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) for large language models…
-
New GFT framework unifies SFT and RL for more stable LLM training
Researchers have introduced Group Fine-Tuning (GFT), a novel framework designed to unify supervised fine-tuning (SFT) and reinforcement learning (RL) for large language models. GFT addresses limitations in traditional S…
-
AgentHER framework boosts LLM agent training with failed trajectory relabeling
Researchers have developed AgentHER, a new framework designed to improve the training of LLM agents by repurposing failed trajectories. The system adapts Hindsight Experience Replay to natural language, identifying alte…
-
LLMs may 'hack' RL training; researchers probe generalization mechanisms
Two new papers explore the complexities of reinforcement learning (RL) in large language models (LLMs). One paper examines how LLMs can be trained to resist RL training by strategically altering their exploration behavi…
-
Compute Aligned Training optimizes LLMs for test-time inference strategies
Researchers have introduced a new training methodology called Compute Aligned Training, designed to better optimize Large Language Models (LLMs) for their performance during inference. Traditional methods like Supervise…
-
Reinforcement learning may be pushing AI models toward alien reasoning, away from human personas
A recent analysis suggests that reinforcement learning (RL) applied after initial model training may significantly alter language model behavior in ways not captured by simple "persona" theories. While supervised fine-t…