FineWeb-Edu CQF model

Classifier quality filtering vulnerable to Wikipedia-style reformatting

Researchers have identified a significant vulnerability in classifier-based quality filtering, a common technique for curating pre-training data for large language models. Their study demonstrates that simple reformatti…