FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language
PulseAugur coverage of FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language — every cluster mentioning FineWeb2: One Pipeline to Scale Them All -- Adapting Pre-Training Data Processing to Every Language across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
New RefinedGAW dataset offers specialized German academic web content
Researchers have introduced RefinedGAW, a new dataset of German academic web content designed to offer greater domain specificity than broad web crawls. The dataset comprises 122 million documents and 314 billion tokens…
-
AI agent finds 1.4M AI-generated docs in Slovak web corpus
Researchers have developed an agentic auditing framework to identify failures in multilingual data processing pipelines like FineWeb2. By using an LLM as a corpus debugger, the agent explored the Slovak portion of FineW…
-
moBERTo: New Portuguese Language Model Enhances NLP Tasks
Researchers have introduced moBERTo, a new Portuguese language model derived from ModernBERT through continued pretraining. This model was trained on 60 billion tokens, incorporating data from FineWeb2 and filtered STEM…