Brief

Researchers have developed ChunkFT, a novel framework designed to significantly reduce the memory required for full-parameter fine-tuning of large language models. This method dynamically activates a working set of parameters, enabling gradient computation on sub-tensors without altering the model architecture. Experiments show ChunkFT can fine-tune models like Llama 3-8B on a single consumer GPU, achieving performance comparable to traditional full fine-tuning while using substantially less memory. AI

Researchers have developed ChunkFT, a new framework designed to make full-parameter fine-tuning of large language models more memory-efficient. This method allows for gradient computation on dynamic subsets of model parameters, reducing the need for extensive GPU memory. Experiments with Llama 3 models demonstrated significant memory savings, enabling fine-tuning on consumer-grade hardware, and achieved performance comparable to or exceeding traditional full fine-tuning methods on various downstream tasks. AI