GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models
Researchers have introduced GenFT, a novel parameter-efficient fine-tuning method for pretrained foundation models. GenFT utilizes a deterministic weight generator conditioned on the model's existing weights to produce task-specific updates. This approach extracts structured patterns from the pretrained weights through row and column transformations with nonlinear activations, aiming to improve performance across various benchmarks in NLP and computer vision. AI
IMPACT Introduces a new technique for more efficient adaptation of large AI models, potentially reducing computational costs for specialized tasks.