Bayesian domain reweighting

New Bayesian method optimizes LLM training data mixtures

Researchers have developed a new Bayesian domain weighting method to optimize the data mixtures used for training large language models (LLMs). This approach infers optimal domain weights from a Dirichlet distribution b…