Stanford Alpaca

Fine-tuning LLMs: Data and evaluation costs often exceed compute

Fine-tuning large language models involves more than just GPU compute costs, with data curation and evaluation often being the largest one-off expenses. The process typically requires multiple attempts, and recurring co…