Concrete Subspace Learning based Interference Elimination for Multi-task Model Fusion
Researchers have developed a new method called Concrete Subspace Learning to improve the fusion of multiple task-specific models derived from a common pre-trained large model. This technique addresses interference issues that arise when combining parameters from different specialized models. By identifying a common low-dimensional subspace, the method aims to retain performance across diverse tasks in the merged model. Experiments in both vision and language domains have shown the effectiveness of this approach. AI