peft
PulseAugur coverage of peft — every cluster mentioning peft across labs, papers, and developer communities, ranked by signal.
8 day(s) with sentiment data
-
180B parameter model Darwin-180B-RSI updated with 0.02% weight change
VIDRAFT has developed Darwin-180B-RSI, a 180-billion-parameter language model that underwent a self-improvement process by updating only 0.02% of its weights. This novel approach allows for efficient model updates while…
-
New ALD-Font model enhances few-shot Chinese font generation
Researchers have developed a new model called ALD-Font for generating Chinese fonts with limited examples. This architecture-level disentangled diffusion model effectively separates content and style, improving both acc…
-
LoRA fine-tuning explained: What's trained, memory use, and dataset impact
This article delves into the specifics of LoRA fine-tuning, explaining that while it trains only a small percentage of a model's parameters, the actual process is more nuanced. It details how LoRA works by adding adapte…
-
Domain Alignment: Adapting LLMs for Specialized Fields
Domain alignment is crucial for adapting general-purpose Large Language Models (LLMs) to specialized fields like healthcare, finance, and education. This process involves training models to understand domain-specific te…
-
LoRA vs. QLoRA: Choosing the Right LLM Fine-Tuning Method
The article compares LoRA and QLoRA, two fine-tuning techniques for large language models. LoRA is suitable when a 16-bit model fits within available GPU memory, while QLoRA is advantageous for memory-constrained scenar…
-
LoRA outperforms QLoRA and full fine-tuning in speed and memory usage
A recent benchmark on an L40S GPU compared three fine-tuning methods for the Qwen2.5-1.5B-Instruct model: full fine-tuning, LoRA, and QLoRA. Full fine-tuning required the most memory, consuming 36.9 GiB with a micro-bat…
-
New repo enables scaled self-distillation with EMA teacher using ZeRO-3 and LoRA
A new repository has been released that enables self-distillation with an Exponential Moving Average (EMA) teacher to function at scale, even with large models. This method utilizes LoRA finetuning, sharing a frozen bac…
-
Free tool helps fine-tune large models on limited GPU memory
A developer has created a free tool called `streamft` to help users fine-tune large language models that exceed their GPU's memory capacity. The tool, available as a browser-based calculator and a Python package, estima…
-
Ollama drops LoRA adapter support, users must merge or use llama.cpp
Ollama has deprecated support for LoRA adapters starting with version 0.34.1, released in September 2026. This change means that Modelfiles containing an ADAPTER line will now result in an error, and Ollama no longer ha…
-
New framework TEPCA enhances LLM reasoning over temporal knowledge graphs
Researchers have developed a new framework called Temporal Evidence-Parameter Co-Alignment (TEPCA) to improve Large Language Models' (LLMs) ability to reason with temporal knowledge graphs. TEPCA addresses limitations i…
-
Guide to Choosing LLM Fine-Tuning Services in 2026
This article provides a guide for selecting a service to fine-tune large language models (LLMs). It covers key considerations such as the model's compatibility, the quality and management of training data, and the imple…
-
New SVC method balances LLM learning and stability
Researchers have developed a new method called Singular-Vector Channel (SVC) for continual learning in Large Language Models (LLMs). This approach addresses the challenge of catastrophic forgetting by selectively updati…
-
Hugging Face Transformers v5.19.0 adds Gemma 2, updates MoE models
The Hugging Face Transformers library has released version 5.19.0, introducing new model additions and significant changes to existing functionalities. Key updates include the EmbeddingGemma2 model, built on Google's Ge…
-
vLLM 0.30.0 mis-scales LoRA adapters, degrading model performance
A bug in vLLM version 0.30.0 causes it to incorrectly scale LoRA adapters, leading to a significant degradation in performance. The issue stems from vLLM ignoring the `rank_pattern` and `alpha_pattern` settings in adapt…
-
New methods tackle LLM backdoor attacks without retraining
Researchers have developed two new methods, NEEDLE and a null-space projection technique, to combat backdoor attacks in Large Language Models (LLMs). NEEDLE, a training-free approach, identifies and suppresses backdoor …
-
MechSparse method guides PEFT selection using mechanistic interpretability
Researchers have developed MechSparse, a novel method for selecting parameters for Parameter-Efficient Fine-Tuning (PEFT) in large language models. Unlike traditional heuristics, MechSparse uses mechanistic interpretabi…
-
New method probes VLM vision encoders for optimal PEFT layer selection
Researchers have developed a new method for selecting optimal layers in vision encoders for vision-language models (VLMs) during parameter-efficient fine-tuning (PEFT). This approach analyzes the statistical properties …
-
Feyn releases MultiMatte, a promptable image background removal model
Feyn has released MultiMatte, a new image background removal model that uses natural language prompts to identify and isolate objects. Built upon Meta's Segment Anything Model 3 (SAM 3), MultiMatte employs low-rank fine…
-
New GAFT Method Enhances Hazard Identification in Off-Road Navigation
Researchers have developed Geo-Anchored Fine-Tuning (GAFT), a novel parameter-efficient method designed to improve hazard identification in off-road navigation. This technique adapts vision foundation models by incorpor…
-
Hugging Face highlights AI advancements in robotics, benchmarking, and fine-tuning techniques
Hugging Face is highlighting several advancements in AI and machine learning. One post details how Strands Agents and LeRobot are being integrated with hardware, showcasing progress in robotics. Another article discusse…