Compress then Merge: From Multiple LoRAs into One Low-Rank Adapter
Researchers have developed a new method called Compress-then-Merge (CtM) to combine multiple Low-Rank Adaptation (LoRA) adapters into a single, more manageable adapter. This approach addresses the fragmentation issue caused by numerous task-specific adapters, which can complicate the reuse and deployment of foundation models. Unlike previous methods that merge adapters first and then compress them, CtM enforces a rank constraint before merging, ensuring the resulting adapter maintains its low-rank structure and efficiency. AI
IMPACT Streamlines the deployment and reuse of specialized foundation models by consolidating multiple adapters into a single, efficient unit.