LLMs are no longer created w/ human data alone. They rely on other models to generate & filter data, evaluate outputs, & guide dev work.
Large language models are increasingly being trained on data generated and filtered by other AI models, rather than solely on human-created data. This shift involves complex interdependencies, with models like Olmo 3 relying on 89 other models and 183 datasets, and Nemotron 3 depending on 273 models and 560 datasets. To help researchers navigate this intricate web of dependencies, the creators have developed a tool called ModSleuth. AI
IMPACT Highlights the growing reliance on synthetic data and complex model interdependencies in LLM development, impacting training efficiency and transparency.