Karanpartap Singh

Research probes pretraining vs. retrieval trade-offs in language models

A new research paper explores the interplay between pretraining and retrieval in language models, investigating how scaling model capacity and pretraining data affects retrieval gains. The study found that retrieval ben…