Parameter-Efficient Fine-Tuning
PulseAugur coverage of Parameter-Efficient Fine-Tuning — every cluster mentioning Parameter-Efficient Fine-Tuning across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
New SSM adapters outperform LoRA for long-context fine-tuning
Researchers have developed a new parameter-efficient fine-tuning (PEFT) method called Hankel Reduced order Model (HRM) adapters, which utilize state space models (SSMs) for long-context fine-tuning. Unlike traditional P…
-
LiMoDE introduces novel two-stage learning for lifelong robot manipulation
Researchers have introduced LiMoDE, a novel two-stage learning scheme designed to improve lifelong robot manipulation capabilities. This approach utilizes a dynamic Mixture-of-Experts (MoE) structure during pre-training…
-
New method predicts LoRA adapter mergeability to avoid performance loss
Researchers have developed a new method called MergeProbe to predict the mergeability of Parameter-Efficient Fine-Tuning (PEFT) updates, specifically for Low-Rank Adaptation (LoRA). This approach aims to forecast whethe…
-
New research explores RL advancements for LLMs and AI agents · 8 sources tracked
Multiple research papers released on arXiv explore advancements in reinforcement learning (RL) for large language models (LLMs) and other AI agents. One paper introduces RiVER, a framework for training LLMs on score-bas…
-
New MODE architecture enhances physics-informed neural networks
Researchers have introduced Manifold-Orthogonal Dual-spectrum Extrapolation (MODE), a novel micro-architecture for adapting physics-informed neural networks (PINNs). MODE addresses limitations in existing methods like S…
-
New PEFT method targets 'flatness preference' for better generalization
Researchers have identified a "flatness preference" in parameter-efficient fine-tuning (PEFT) methods, suggesting that a small subset of dimensions significantly impacts generalization. They propose Flatness Preference …
-
New SMoA Adapter Boosts LLM Fine-Tuning Efficiency
Researchers have introduced SMoA, a novel Spectrum Modulation Adapter designed to enhance parameter-efficient fine-tuning (PEFT) for large language models. Unlike traditional methods like Low-Rank Adaptation (LoRA) whic…
-
New Deep Reprogramming Distillation framework enhances medical AI models
Researchers have introduced a new framework called Deep Reprogramming Distillation (DRD) to address the challenges of adapting large medical foundation models for specific downstream tasks. DRD utilizes a novel reprogra…
-
Mantis framework offers efficient Mamba-native tuning for 3D point cloud models
Researchers have introduced Mantis, a novel framework for parameter-efficient fine-tuning (PEFT) specifically designed for Mamba-based 3D point cloud foundation models. Existing PEFT methods struggle with Mamba's state-…