PulseAugur
EN
LIVE 10:44:29

New Hard-Routed MoR-LoRA framework efficiently composes reasoning adapters

By PulseAugur Editorial · [2 sources] ·

Researchers have developed a new framework called Hard-Routed MoR-LoRA for composing independently trained LoRA adapters into a single large language model. This method uses a two-stage process where domain-specific LoRA adapters are trained as reasoning experts, and then a lightweight router is trained to select exactly one expert per token via hard top-1 routing. Experiments across various benchmarks and model scales demonstrate that this approach preserves expert behavior while requiring significantly fewer trainable parameters compared to soft-routing mixture baselines. AI

IMPACT This method could enable more efficient adaptation of large language models by composing specialized, independently trained adapters.

RANK_REASON Research paper detailing a new method for composing LoRA adapters. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →

New Hard-Routed MoR-LoRA framework efficiently composes reasoning adapters

COVERAGE [2]

  1. arXiv cs.AI TIER_1 English(EN) · Seyed Alireza Molavi, Zhan Su, Yan Hu, Peyman Sheikholharam Mashhadi, Stefan Byttner, Prayag Tiwari ·

    Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

    arXiv:2606.31413v1 Announce Type: new Abstract: Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared. A common approach is to use MoE-style routing over …

  2. arXiv cs.LG TIER_1 English(EN) · Prayag Tiwari ·

    Learning to Select, Not Relearn: Hard-Routed Mixtures of Reasoning LoRAs

    Composing independently trained LoRA adapters into a single large language model is useful for multi-domain adaptation, especially when the original training data cannot be shared. A common approach is to use MoE-style routing over LoRA experts, but for frozen pretrained adapters…

RELATED ENTITIES

RELATED TOPICS