Researchers have developed Infini-News, a toolkit and index designed to provide efficient access to over 1.3 billion news articles from the Common Crawl archive. This new resource includes cleaned text, structured metadata, language detection, and geographic attribution for each article. The system utilizes Infini-gram indexes, enabling researchers to search the entire archive for text patterns in under a second, thereby facilitating large-scale media research. AI
IMPACT Lowers the barrier for computational social science and NLP research by providing efficient access to a massive news corpus.
RANK_REASON Publication of an academic paper detailing a new toolkit and dataset for NLP research. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
