Apple ML Research scales flow maps for faster language generation

Apple Machine Learning Research has published a paper detailing advancements in scaling Categorical Flow Maps (CFMs) for language modeling. The research team successfully trained a 1.7 billion parameter flow model on 2.1 trillion tokens and distilled it into a CFM capable of generating diverse, high-quality text in as few as 4 inference steps. This approach maintains near-data-level token entropy and shows promise for discrete data generation, addressing previous limitations in scalability. AI

IMPACT This research could lead to more efficient and faster text generation models, potentially impacting how language models are trained and deployed.

RANK_REASON Publication of a research paper detailing a new method for language modeling. [lever_c_demoted from research: ic=1 ai=1.0]

Read on Apple Machine Learning Research →

• Amitis Shidani
• Anastasiia Filippova
• Apple Inc.
• CAR-Flow
• Categorical Flow Maps
• Conference on Neural Information Processing Systems
• Louis Béthune
• Marco Cuturi
• Oscar Davis
• Pierre Ablin
• University of Oxford
• Victor Turrisi

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →