Cogs

New FORA technique preserves LLM capabilities during fine-tuning

Researchers have developed a new fine-tuning technique called FORA (Function-space Orthogonal Residual Adaptation) that aims to preserve a large language model's existing capabilities while adapting it to new tasks. Unl…