supervised fine-tuning
PulseAugur coverage of supervised fine-tuning — every cluster mentioning supervised fine-tuning across labs, papers, and developer communities, ranked by signal.
- used by Group Relative Policy Optimization 90%
- instance of Direct Preference Optimization 90%
- instance of Grpo 70%
- affiliated with Direct Preference Optimization: Your Language Model is Secretly a Reward Model 70%
- used by alphaXiv 70%
- instance of Gotit.pub 70%
- used by CatalyzeX 70%
- used by Direct Preference Optimization: Your Language Model is Secretly a Reward Model 70%
- instance of alphaXiv 70%
- instance of Direct Preference Optimization: Your Language Model is Secretly a Reward Model 70%
- used by Gotit.pub 70%
- instance of CatalyzeX 70%
21 day(s) with sentiment data
-
AI model evaluation evolves from token prediction to complex workflows
The evaluation of frontier AI models has evolved through distinct eras, moving from simple token prediction and log-likelihood grading to assessing instruction following, then to verifying math, code, factuality, and sa…
-
BiGrounder enhances GUI grounding with bidirectional training
Researchers have developed BiGrounder, a novel approach to GUI grounding that maps natural language instructions to precise operational coordinates for autonomous agents. Unlike previous methods that relied solely on fi…
-
New theory explains LLM capability loss during fine-tuning
Researchers have developed a new theoretical framework to explain Pervasive Capability Degradation (PCD) in large language models, a phenomenon where fine-tuning for specific tasks degrades general abilities. The theory…
-
Study questions value of RLVR after SFT for AI training
A new study published on arXiv investigates the effectiveness of Reinforcement Learning from Verifiable Rewards (RLVR) when used after Supervised Fine-Tuning (SFT) for AI model training. Researchers developed a benchmar…
-
Research reveals LoRA dynamics in SFT vs RL, proposes tailored configurations
A new research paper published on arXiv details the spectral and subspace dynamics of Low-Rank Adaptation (LoRA) during supervised fine-tuning (SFT) and reinforcement learning (RL). The study found that RL concentrates …
-
Personal AI Assistant Enhanced with LoRA on Home Hardware
This article details a personal project to enhance a large language model (LLM) using LoRA (Low-Rank Adaptation) on home hardware. The author describes the process of creating a personal assistant, focusing on its evolv…
-
New AI framework LegalEvolver optimizes legal judgment prediction queries
Researchers have developed LegalEvolver, a novel framework designed to enhance legal judgment prediction by optimizing retrieval queries. This system uses a rubric-guided self-evolution process, where initial queries ar…
-
New framework shapes AI models via internal representations
Researchers have introduced Mechanistic Shaping, a novel framework designed to guide AI models by influencing their internal representations during training. This method aims to ensure models adhere to specific rules re…
-
Domain Alignment: Adapting LLMs for Specialized Fields
Domain alignment is crucial for adapting general-purpose Large Language Models (LLMs) to specialized fields like healthcare, finance, and education. This process involves training models to understand domain-specific te…
-
New MicroMouse framework masks harmful supervision in LLM reasoning
Researchers have developed a new framework called MicroMouse to improve the training of large language models (LLMs) by identifying and masking "harmful supervision" in long Chain-of-Thought (CoT) reasoning processes. I…
-
ViKernel blocks AI agent backdoor payloads missed by standard scans
A new security kernel called ViKernel, developed by VIDRAFT, effectively blocks backdoor payloads in AI agents. This comes after research demonstrated that even a small percentage of poisoned pretraining data can embed …
-
Steering vectors show promise as LLM training signal
Researchers have explored the use of steering vectors (SVs) as a training signal for post-training large language models (LLMs). Their framework investigates SV injection during supervised learning, including supervised…
-
New research formalizes controllable user simulation as causal inference
A new research paper published on arXiv introduces a novel approach to controllable user simulation for evaluating conversational agents. The authors formalize this simulation as a causal inference problem, highlighting…
-
New metric "Expected Harm" challenges LLM safety evaluations
A new research paper proposes a metric called "Expected Harm" to re-evaluate how the safety of large language models is assessed. The authors argue that current methods overemphasize the severity of harmful outputs whil…
-
Nemotron Labs releases Dory1.0-4B looped language model
Nemotron Labs has released Dory1.0-4B, a 4-billion-parameter language model that utilizes a novel looped architecture. This design allows the model to achieve improved accuracy and token efficiency by applying a shared …
-
New framework P4IR boosts LLM accuracy in automated code compliance
Researchers have developed P4IR, a novel two-stage framework designed to enhance the accuracy of large language models (LLMs) in generating computer-processable building regulations. The first stage employs supervised f…
-
New RL Framework Improves Multi-Image Composition Consistency
Researchers have developed DeCoRL, a reinforcement learning framework designed to enhance multi-image composition (MICo) capabilities in large-scale image editing models. This approach decomposes the overall MICo consis…
-
New training method improves language model introspection accuracy
Researchers have developed a new method called "interventional training" to improve the accuracy of language model introspection. Standard supervised fine-tuning (SFT) can lead to models relying on correlated but causal…
-
New Merge and Branch method boosts LLM fine-tuning efficiency for recommendations
Researchers have developed a new method called Merge and Branch (M&B) to make supervised fine-tuning of large language models (LLMs) more efficient for recommendation systems. M&B compresses historical item information …
-
AI models develop 'self-concept' through fine-tuning, research shows
A new research paper explores how language models develop a sense of self-concept through supervised fine-tuning. By training models on declarative statements about their identity, researchers found that models adopted …