Can LoRA Fusion Support Cross-Domain Tasks in Cloud-Edge Collaboration?
Researchers have developed a new framework called prune-train-recover to enable local LoRA training on pruned models for cloud-edge collaboration. This approach addresses privacy concerns by allowing LoRA adapters to be trained on edge devices without uploading sensitive domain data. The study also introduced MMLU-CD, a cross-domain benchmark, and found that existing LoRA fusion methods perform poorly on such tasks, often underperforming the base LLM. To combat this, a conflict-resolution module, LoRA-CR, was proposed, which improved performance by up to 3.8% by mitigating parameter conflicts. AI
IMPACT This research could enable more effective and privacy-preserving AI model adaptation in distributed cloud-edge environments.