MLorc

New methods enhance LLM adaptation with efficient, structured low-rank tuning

Researchers have introduced MLorc, a novel method for memory-efficient adaptation of large language models that compresses parameter momentum during training. This approach aims to reduce memory demands without sacrific…