Lora
PulseAugur coverage of Lora — every cluster mentioning Lora across labs, papers, and developer communities, ranked by signal.
- used by Vít 90%
- instance of Low Rank Adaptation 90%
- used by large-language models 70%
- instance of Direct Preference Optimization 70%
- used by magazine 70%
- used by Glue 70%
- used by supervised fine-tuning 70%
- developed large-language models 70%
- used by Bert 70%
- used by Dopravní podnik Ostrava 70%
- used by Transformer Reinforcement Learning 70%
- instance of peft 70%
- 2026-05-12 research_milestone A paper is published detailing findings on parameter placement in LoRA for fine-tuning. 来源
16 天有情绪数据
-
开发者使用 QLoRA 在 3GB GPU 上微调 LLM
开发者可以使用 QLoRA 和 NF4 量化等技术，在仅需 3 GB GPU 内存的消费级硬件上微调 TinyLlama 等大型语言模型。此过程仅训练模型的一小部分参数，显著降低了计算需求。尽管该过程可能很复杂，在调试、提示格式化和依赖管理方面存在挑战，但它为独立开发者构建复杂的 AI 应用程序提供了一条途径。
-
New AR1-ZO method boosts LoRA fine-tuning with Zeroth-Order optimization
Researchers have developed AR1-ZO, a novel method for fine-tuning large language models using Zeroth-Order optimization and Low-Rank Adaptation (LoRA). This technique addresses the challenge of effectively increasing Lo…
-
Qwen 3.7 支持可调式审查，NVIDIA 微调机器人视频生成
Qwen 发布了其语言模型 3.7 版本，该版本包含一个可修改的政治审查专用电路，且修改后不损失事实知识。NVIDIA 的 Cosmos Predict 2.5 模型现在可以使用高效的 LoRA/DoRA 方法进行机器人视频生成的微调。此外，新的 HRM-Text 模型为预训练基础模型提供了更易于访问且成本效益更高的方法。
-
NamelessDogLab releases character LoRAs for AI web UIs
NamelessDogLab is releasing character LoRAs for web UI use to Patreon supporters. These LoRAs are designed to enable the creation of specific characters within AI-driven web interfaces. The project encourages support th…
-
New infrastructure enables one base AI model to serve millions of LoRA policies
Researchers have developed a new infrastructure that allows a single base AI model to efficiently serve millions of LoRA (Low-Rank Adaptation) policies. This approach avoids the need to copy weights for each policy, sig…
-
Fine-tuning Qwen2.5 with LoRA yields structured, not necessarily correct, outputs
This article explores the process of fine-tuning the Qwen2.5 model using the LoRA technique. It demonstrates that while fine-tuning can lead to more structured outputs, this does not necessarily equate to improved reaso…
-
LLM Fine-Tuning Explained: SFT, RAG, and Data Preparation
This blog post explains the process and necessity of fine-tuning large language models (LLMs) for specific tasks. It differentiates fine-tuning from Retrieval-Augmented Generation (RAG), stating that fine-tuning is best…
-
EverAnimate method improves long-form animated video generation
Researchers have developed EverAnimate, a novel post-training method designed to improve the generation of long-form animated videos. This technique addresses issues like visual quality degradation and inconsistent char…
-
PreFT method boosts LLM serving throughput with prefill-only finetuning
Researchers have developed PreFT, a novel parameter-efficient finetuning method designed to improve the efficiency of serving personalized large language models. PreFT optimizes for serving throughput by applying adapte…
-
PEML method optimizes LLM prompts and weights for multi-task learning
Researchers have introduced PEML, a new method for parameter-efficient multi-task learning in large language models. PEML optimizes both continuous prompts and model weights simultaneously, addressing limitations of exi…
-
New GPart method offers efficient LLM fine-tuning
Researchers have introduced GPart, a novel parameter-efficient fine-tuning method that bypasses the low-rank bottleneck inherent in LoRA. GPart utilizes a single isometric partition matrix to map a low-dimensional train…
-
LoRA parameter placement impacts GRPO fine-tuning, not SFT
Researchers have investigated the parameter placement problem within Low-Rank Adaptation (LoRA) for fine-tuning large language models. Their study reveals that for Supervised Fine-Tuning (SFT), the specific placement of…
-
New research explores efficient LLM alignment and federated fine-tuning
Researchers are developing new methods for efficient large language model (LLM) alignment and fine-tuning. One approach, P2D, uses task-sensitive attention heads to guide data selection and parameter pruning, achieving …
-
新研究分解LoRA架构，识别出路由器重写是关键性能驱动因素
研究人员开发了一种方法，将进化式LoRA混合架构分解为三个关键组成部分：路由器重写、每域评估范围和生命周期机制。他们在约1.5亿参数的基底上进行的实验表明，路由器重写是大部分性能提升的原因，具体表现为+0.0426 nat的平衡对数困惑度（log-PPL）增益。然而，生命周期机制被发现对性能有净负面影响，而评估范围在种子分辨率下未显示出显著影响。
-
New AdaPaD method improves PEFT efficiency for large language models
Researchers have introduced AdaPaD, a novel method for efficiently fine-tuning large language models using Parameter-Efficient Fine-Tuning (PEFT). AdaPaD trains all rank-1 components simultaneously, with each component …
-
Fashion Florence model extracts structured clothing attributes
Researchers have developed Fashion Florence, a vision-language model based on Florence-2, specifically fine-tuned for extracting structured fashion attributes from images. This model can generate a JSON object detailing…
-
EvoPref算法通过进化优化增强语言模型对齐
研究人员开发了EvoPref，这是一种新颖的多目标进化算法，旨在改进大型语言模型（LLM）的对齐。与可能导致偏好崩溃和狭窄行为模式的传统基于梯度的方法不同，EvoPref维护了针对有用性、无害性和诚实性进行优化的适配器多样化种群。这种方法显著增强了偏好覆盖范围并降低了崩溃率，同时实现了具有竞争力的对齐质量，确立了进化优化作为多样化LLM对齐的可行范式。
-
LoRA fine-tuning reduces LLM parameter updates
Low-Rank Adaptation (LoRA) is a technique for efficiently fine-tuning large language models. Instead of modifying all model weights, LoRA freezes the original weights and introduces small, trainable matrices to learn ad…
-
LoRA fine-tuning explained with matrix-level detail
This article provides a detailed, number-by-number explanation of how LoRA (Low-Rank Adaptation) works for fine-tuning large language models. It aims to go beyond simply stating what LoRA achieves and instead illustrate…
-
LoRA Explained: Mathematical Intuition Behind Low-Rank Adaptation
This article delves into the mathematical underpinnings of Low-Rank Adaptation (LoRA), a technique used for efficient fine-tuning of large language models. It explains how LoRA leverages the concept of low intrinsic dim…