Parameter-Efficient Fine-Tuning
PulseAugur coverage of Parameter-Efficient Fine-Tuning — every cluster mentioning Parameter-Efficient Fine-Tuning across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
New method boosts efficiency of point cloud transformers
Researchers have introduced Position Anchor Tuning (PAT), a novel parameter-efficient fine-tuning method designed to improve the inference efficiency of pre-trained point cloud transformers. PAT addresses computational …
-
New DROS method enhances foundation model adaptation for remote sensing
Researchers have developed a new method called Domain-aware Relaxed Orthogonal Subspace adaptation (DROS) to improve the efficiency of fine-tuning large foundation models for remote sensing tasks. This approach addresse…
-
LLM fine-tuning made accessible with LoRA and Unsloth · 2 sources tracked
Two articles detail methods for fine-tuning large language models (LLMs) using parameter-efficient techniques. The first explains how to use LoRA (Low-Rank Adaptation) with Unsloth to fine-tune a 7B LLM, demonstrating a…
-
New benchmarks and methods advance AI's 3D scene understanding capabilities · 3 sources tracked
Three new research papers introduce novel benchmarks and methods for improving 3D scene understanding in AI. SceneBench offers a hierarchical dataset with detailed annotations for evaluating vision-language models on sp…
-
New SVGD framework enhances AI model fine-tuning with geometry awareness
Researchers have developed a new framework for parameter-efficient fine-tuning of large pre-trained models that leverages the geometric structure of low-rank manifolds. This approach utilizes Stein Variational Gradient …
-
New HiVe framework enhances LLM multitask learning with adaptive prompts
Researchers have introduced HiVe, a novel prompt tuning framework designed to enhance multitask learning in large language models (LLMs). Unlike existing methods that use static or fixed hierarchical prompt structures, …
-
New benchmarks and methods tackle LLM hallucinations across modalities and domains
Researchers are developing new methods and benchmarks to detect and mitigate hallucinations in large language models (LLMs) across various modalities and domains. OmniHallu offers a unified framework for detecting hallu…
-
FrameFT reduces fine-tuning memory footprint with sparse coefficients
Researchers have introduced FrameFT, a novel Parameter-Efficient Fine-Tuning (PEFT) strategy designed to reduce the memory footprint of fine-tuning large transformer models. Unlike existing methods like LoRA, FrameFT mo…
-
New FAN-LoRA method improves medical image segmentation for foundation models
Researchers have developed FAN-LoRA, a new method for adapting vision foundation models like the Segment Anything Model (SAM) to medical imaging domains. Existing methods struggle with domain shifts, leading to performa…
-
PEFT boosts LLM hate speech detection in Roman Urdu to over 93% F1
A new research paper explores the effectiveness of Parameter-Efficient Fine-Tuning (PEFT) methods, specifically Low-Rank Adaptation (LoRA), for hate speech detection in Roman Urdu. The study found that while zero-shot i…
-
New CKAA framework boosts continual learning model robustness
Researchers have introduced CKAA, a novel framework designed to improve the robustness of continual learning models against misleading task identifications. The framework incorporates Dual-level Knowledge Alignment (DKA…
-
New sMuon method enhances low-rank fine-tuning for neural networks
Researchers have developed a new method called sMuon to improve parameter-efficient fine-tuning (PEFT) of neural networks. This technique addresses the incompatibility between the Muon optimizer and the common LoRA PEFT…
-
Wavelet Fine-Tuning (WaveFT) offers advanced parameter efficiency for AI models
Researchers have introduced Wavelet Fine-Tuning (WaveFT), a novel method for parameter-efficient fine-tuning (PEFT) that utilizes sparsity in the wavelet domain of weight matrices. Unlike existing methods like LoRA, Wav…
-
New methods accelerate Vision Transformer adaptation for edge devices
Researchers have developed new methods for adapting Vision Transformers (ViTs) to specific tasks more efficiently. One approach uses genetic programming to evolve layer-specific scalar functions that approximate normali…
-
LoRA enables efficient fine-tuning of large language models
LoRA (Low-Rank Adaptation) is a technique that allows for efficient fine-tuning of large language models. It works by freezing the original model's weights and injecting smaller, trainable matrices into specific layers,…
-
LoRA fine-tuning offers startups efficient LLM adaptation
LoRA (Low-Rank Adaptation) is a technique that significantly reduces the computational resources needed for fine-tuning large-language models. While often described as a cheaper method, LoRA offers more than just cost s…
-
HyPASE framework uses hyperbolic geometry for efficient LALM fine-tuning
Researchers have developed HyPASE, a novel framework that utilizes hyperbolic geometry for parameter-efficient fine-tuning of Large Audio-Language Models (LALMs) for Speech Emotion Recognition (SER). Unlike traditional …
-
New research tackles domain adaptation for V2X collaborative perception
Two new research papers introduce advanced techniques for domain-generalized adaptive semantic communication in collaborative perception systems, particularly for Vehicle-to-Everything (V2X) applications. The first pape…
-
New Z-PEFT method detects backdoors in fine-tuned AI models
Researchers have developed Z-PEFT, a novel method for detecting backdoors in Parameter-Efficient Fine-Tuning (PEFT) models. This approach utilizes canonical spectral signatures from model weights to identify malicious m…
-
MoPET method uses mixture-of-experts for medical image classification
Researchers have developed MoPET, a novel parameter-efficient fine-tuning (PEFT) method that utilizes a mixture-of-experts (MoE) approach for medical image classification. MoPET addresses the issue of negative transfer …