Lora
PulseAugur coverage of Lora — every cluster mentioning Lora across labs, papers, and developer communities, ranked by signal.
- used by Vít 90%
- instance of Low Rank Adaptation 90%
- used by large-language models 70%
- used by peft 70%
- instance of Direct Preference Optimization 70%
- used by Glue 70%
- used by magazine 70%
- used by supervised fine-tuning 70%
- developed large-language models 70%
- used by Bert 70%
- used by Dopravní podnik Ostrava 70%
- used by Transformer Reinforcement Learning 70%
- 2026-05-12 research_milestone A paper is published detailing findings on parameter placement in LoRA for fine-tuning. 来源
16 天有情绪数据
-
AI artist masters Gérôme's fini surface technique with advanced LoRA training
An AI artist has developed a LoRA model capable of replicating Jean-Léon Gérôme's signature "fini surface" technique. This involved three iterative training rounds to blend academic painting precision with machine learn…
-
Paired bootstrapping is key for AI model evaluation, article explains
A technical analysis explains the statistical necessity of paired bootstrapping in evaluating AI model performance, particularly when comparing a baseline system against a trained LoRA model. The author demonstrates tha…
-
MatryoshkaLoRA enhances LLM fine-tuning with hierarchical low-rank representations
Researchers have introduced MatryoshkaLoRA, a novel framework for fine-tuning large language models that improves efficiency and performance. This method uses a hierarchical approach to low-rank representations, inserti…
-
New Bayesian fine-tuning method enhances model uncertainty quantification
Researchers have developed a new framework for parameter-efficient Bayesian fine-tuning of large models. This method quantifies uncertainty effectively within very low-dimensional parameter spaces, addressing limitation…
-
临床 AI 在 AMD 硬件上微调，绕过 CUDA 依赖
一个项目已成功在 AMD 硬件和 ROCm 上微调了临床 AI 模型 MedQA，证明了在没有 NVIDIA 的 CUDA 的情况下也可以进行高级 AI 开发。微调过程使用了 Qwen3-1.7B 模型和 MedMCQA 数据集，仅在 AMD Instinct MI300X 上花费了五分钟就取得了成果。这项工作突显了 Hugging Face 生态系统与 ROCm 的兼容性，可能拓宽 AI 开发工具的可及性。
-
LoRA rank allocation fails in RL fine-tuning, study finds
A new study on the Qwen 2.5 1.5B model reveals that adaptive rank allocation techniques, effective in supervised fine-tuning, do not translate to reinforcement learning with Group Relative Policy Optimization (GRPO). Re…
-
新的Diff-SAE方法在检测语言模型后门方面表现出色
研究人员开发了一种使用稀疏自编码器（SAE）的新方法来检测语言模型中的后门攻击。他们的差分SAE（Diff-SAE）架构在隔离恶意特征方面比Crosscoders更有效。这种方法对于通过提供识别和减轻模型操纵的工具来增强AI安全至关重要。
-
新的防御框架应对多语言提示注入攻击
研究人员开发了MIPIAD，一个用于防御多语言大型语言模型系统中间接提示注入攻击的防御框架。该框架结合了使用LoRA微调的Qwen2.5-1.5B模型、TF-IDF词汇特征以及集成学习方法。在英语和孟加拉语上进行评估，MIPIAD使用混合集成达到了0.9205的高F1分数，使用提升集成达到了0.9378的AUROC，证明了其在缩小跨语言差距方面的有效性。
-
Transformer memory geometry explains confident hallucinations in LLMs
Researchers have developed a new geometric framework to understand two failure modes in language models: conflict and hallucination. They propose that learned facts form attractor basins in the model's hidden-state spac…
-
New adapter TFM-Retouche improves tabular foundation models without fine-tuning
Researchers have developed TFM-Retouche, a novel adapter designed to enhance tabular foundation models (TFMs) without requiring computationally expensive full fine-tuning. This lightweight, architecture-agnostic adapter…
-
HCInfer system enables LLMs on resource-constrained devices with error compensation
Researchers have developed HCInfer, a novel inference system designed to enable large language models (LLMs) to run efficiently on devices with limited memory. This system offloads parts of the model's compensation mech…
-
新的AS-LoRA方法提高了联邦学习的隐私性
研究人员开发了AS-LoRA，一种用于隐私保护联邦学习中LoRA组件自适应选择的新型框架。该方法通过允许每一层独立选择其活动组件并在通信轮次中调整这些选择来解决此类设置中常见的聚合错误。AS-LoRA在不增加隐私成本的情况下，理论上提高了收敛速度和准确性，并在GLUE和SQuAD等基准测试中取得了显著的进步。
-
New DLoR framework proves universal approximation with sparse diagonal components
Researchers have introduced a new framework called Structural Correspondence for neural networks that use parameter-efficient low-rank structures. This framework demonstrates that augmenting low-rank layers with a minim…
-
DPO vs SimPO: Preference tuning methods compared for LLM training
A recent analysis highlights a critical discrepancy in preference tuning methodologies for large language models, specifically comparing Direct Preference Optimization (DPO) and Simplified Preference Optimization (SimPO…
-
LoRA fine-tuning: Style learning or pattern memorization?
A recent analysis explores whether fine-tuning a LoRA adapter on a specific writing style, like "Tenacious-style" sales emails, results in genuine style imitation or mere memorization of augmented patterns. The study fo…
-
New research links optimizer choice to reduced forgetting in LLM finetuning
Researchers have explored the impact of optimizer consistency during the fine-tuning of large language models. One study suggests that using the same optimizer for both pre-training and fine-tuning leads to less knowled…
-
LoRA 微调详解：为什么低秩能有效适配大语言模型
本文解释了大语言模型微调的内在低秩假设，详细说明了 LoRA 等技术如何在不改变原始权重的情况下适配模型。文章阐明，LoRA 的表达性更新仅限于秩 r 的子空间，这意味着如果更高的秩超过了任务的内在秩，性能不一定会提高。作者提供了一个可运行的脚本和实证结果，以展示 LoRA 的秩如何影响其拟合必要更新子空间的能力，并表明过度参数化会导致噪声。
-
PACZero 实现具有可用效用的 PAC-私有语言模型微调
研究人员开发了 PACZero，一种新颖的大型语言模型微调方法，可提供强大的隐私保证。该方法利用梯度的符号量化来实现一种隐私机制，在这种机制下，成员推断攻击的成功率不高于随机猜测。PACZero 在 SST-2 和 SQuAD 等标准基准测试中表现出具有竞争力的性能，即使在零互信息的情况下，在高隐私设置下也优于先前的方法。
-
Fine-tuned small language models outperform LLMs in Windows event log analysis
A new paper explores the use of small language models (SLMs) for analyzing Windows event logs, offering a more resource-efficient alternative to large language models (LLMs). Researchers developed a synthetic dataset wi…
-
DomLoRA method places single adapter at dominant module for efficient fine-tuning
Researchers have developed a new method called DomLoRA for parameter-efficient fine-tuning of large language models. This technique identifies a single "dominant adaptation module" within a model where placing a low-ran…