New dataset distillation methods improve synthetic data generation for ML training

Researchers have developed a new method for dataset distillation, a technique that creates a small synthetic dataset to replace a large real one for training machine learning models. This novel approach, called "Dataset Distillation via Drifting," focuses on approximating the per-class data distribution rather than solely optimizing for a classifier. By "drifting" synthetic data in a feature extractor's latent space, the method aims to better cover the real data distribution and improve classification performance. Additionally, a related technique, Model Distribution-Aware Multimodal Dataset Distillation (MDA-DD), synthesizes compact image-text datasets for vision-language pre-training, showing significant improvements in retrieval tasks with substantially fewer synthetic data points. AI

IMPACT These advancements in dataset distillation could significantly reduce the computational cost and data requirements for training large machine learning models, particularly in multimodal domains.

RANK_REASON The cluster contains two research papers detailing new methods for dataset distillation.

Read on Hugging Face Daily Papers →

• Flickr30K
• MDA-DD
• Model Distribution-Aware Multimodal Dataset Distillation
• Ms Coco
• arXiv
• Dataset Distillation via Drifting
• George Cazenavette V
• Hugging Face

• paper
• model release

AI-generated summary · Google Gemini · from 2 sources. How we write summaries →