PulseAugur
EN
LIVE 00:41:44

Open-source tool Cull updated for local dataset curation

A developer has updated an open-source, locally-run tool called Cull designed for dataset curation. The tool can scrape, sort, and caption images and videos from various sources like Civitai, X, and Reddit. It incorporates features such as deduplication, quality and topic-relevance scoring, watermark detection, and automatic captioning, with options to export datasets to local files or HuggingFace. AI

IMPACT Enhances efficiency for AI developers and researchers managing large image and video datasets.

RANK_REASON Update to an existing open-source tool for dataset management.

Read on r/StableDiffusion →

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →

Open-source tool Cull updated for local dataset curation

COVERAGE [1]

  1. r/StableDiffusion TIER_2 English(EN) · /u/Compunerd3 ·

    Updated my tool that scrapes,sorts,captions images/videos for datasets. It's open source and runs locally

    <!-- SC_OFF --><div class="md"><p>I built Cull a few months ago for some large scale dataset curation projects (300k+ images/videos).</p> <p>Point it at Civitai, X, Reddit, Discord, or any URL that gallery-dl or yt-dlp knows. It queues everything, runs a vision model (or multiple…