supervised fine-tuning
PulseAugur coverage of supervised fine-tuning — every cluster mentioning supervised fine-tuning across labs, papers, and developer communities, ranked by signal.
7 天有情绪数据
-
New SFT objectives outperform NLL for capable LLMs
Researchers have explored alternative objectives for supervised fine-tuning (SFT) of large language models, moving beyond the standard negative log likelihood (NLL). Their study, involving extensive experiments across v…
-
Anyscale launches skill to automate LLM post-training runs
Anyscale has introduced a new Anyscale Agent Skill designed to simplify and automate the process of generating LLM post-training runs. This skill assists users in selecting the most appropriate post-training method, suc…
-
Off-model SFT degrades AI capabilities by forcing unfamiliar reasoning styles
Researchers have found that Supervised Fine-Tuning (SFT) using outputs from a different AI model can significantly degrade the capabilities of the trained model. This degradation appears to be linked to the model adopti…
-
LLM Fine-Tuning Explained: SFT, RAG, and Data Preparation
This blog post explains the process and necessity of fine-tuning large language models (LLMs) for specific tasks. It differentiates fine-tuning from Retrieval-Augmented Generation (RAG), stating that fine-tuning is best…
-
New method searches data recipes for optimal AI model fine-tuning
Researchers have developed a new method for supervised fine-tuning (SFT) data selection, moving beyond simple instance ranking to a "data recipe search" approach. This technique uses a library of operators like filterin…
-
LoRA parameter placement impacts GRPO fine-tuning, not SFT
Researchers have investigated the parameter placement problem within Low-Rank Adaptation (LoRA) for fine-tuning large language models. Their study reveals that for Supervised Fine-Tuning (SFT), the specific placement of…
-
New training method combats LLM diversity loss
Researchers have developed a new method called annotation-anchored training to address semantic mode collapse in large language models. This technique involves pretraining models on documents paired with semantic annota…
-
LLM reasoning emerges via Inverse Tree Freezing, improving multi-step thinking
Researchers have developed a new framework called Inverse Tree Freezing to understand how large language models (LLMs) achieve complex reasoning. This model views the LLM's learning process as a random walk on a 'Concep…
-
LoRA fine-tuning: Style learning or pattern memorization?
A recent analysis explores whether fine-tuning a LoRA adapter on a specific writing style, like "Tenacious-style" sales emails, results in genuine style imitation or mere memorization of augmented patterns. The study fo…
-
Pest-Thinker uses RL to help MLLMs reason like entomologists
Researchers have developed Pest-Thinker, a novel reinforcement learning framework designed to enhance the reasoning capabilities of multimodal large language models (MLLMs) for agricultural pest identification. This sys…
-
New methods enhance on-policy distillation for LLMs
Researchers have developed new methods to improve the efficiency and stability of on-policy distillation (OPD) for large language models. One approach, vOPD, uses a control variate baseline derived from the reverse KL d…
-
ProFit method enhances LLM fine-tuning by prioritizing high-value signals
Researchers have developed a new supervised fine-tuning (SFT) method called ProFit, designed to improve the alignment of Large Language Models (LLMs) with human intent. ProFit addresses the issue of overfitting to speci…
-
Researchers use RL to improve MLLM regression on imbalanced data
Researchers have developed a new framework to improve how multimodal large language models (MLLMs) handle numerical regression tasks, particularly those with imbalanced data distributions. Existing training methods ofte…
-
Judge-R1 framework enhances legal document generation with agentic information retrieval
Researchers have developed Judge-R1, a new framework to improve the automated drafting of legal judgment documents. This system uses an agentic approach to collect relevant legal information and a reinforcement learning…
-
AI model finetuning mostly idempotent, DPO can amplify traits
A guide explores advanced techniques for post-training large language models, focusing on Supervised Fine-Tuning (SFT), Direct Preference Optimization (DPO), and Group Relative Policy Optimization (GRPO). These methods …
-
New DoTS framework synthesizes SFT and RLVR LLM capabilities at inference time
Researchers have developed a novel post-hoc framework called Decoupled Test-time Synthesis (DoTS) to integrate Supervised Fine-Tuning (SFT) and Reinforcement Learning from Human Feedback (RLHF) for large language models…
-
New GFT framework unifies SFT and RL for more stable LLM training
Researchers have introduced Group Fine-Tuning (GFT), a novel framework designed to unify supervised fine-tuning (SFT) and reinforcement learning (RL) for large language models. GFT addresses limitations in traditional S…
-
AgentHER framework boosts LLM agent training with failed trajectory relabeling
Researchers have developed AgentHER, a new framework designed to improve the training of LLM agents by repurposing failed trajectories. The system adapts Hindsight Experience Replay to natural language, identifying alte…
-
LLMs may 'hack' RL training; researchers probe generalization mechanisms
Two new papers explore the complexities of reinforcement learning (RL) in large language models (LLMs). One paper examines how LLMs can be trained to resist RL training by strategically altering their exploration behavi…
-
Compute Aligned Training optimizes LLMs for test-time inference strategies
Researchers have introduced a new training methodology called Compute Aligned Training, designed to better optimize Large Language Models (LLMs) for their performance during inference. Traditional methods like Supervise…