PulseAugur
中
实时 08:39:32

Google DeepMind unveils Decoupled DiLoCo for resilient AI model training

作者 PulseAugur 编辑部 · [6 个来源] ·

Google DeepMind has introduced Decoupled DiLoCo, a novel approach to training advanced AI models that enhances resilience and flexibility across data centers. This system can train models like Google's 12B Gemma model across geographically dispersed regions using low-bandwidth networks and can even mix different generations of hardware, such as TPU6e and TPUv5p. Decoupled DiLoCo is designed to be self-healing, isolating and continuing training through artificial hardware failures and reintegrating units when they come back online, addressing the synchronization issues that typically stall AI training. AI

影响 Enables more robust and flexible large-scale AI model training, potentially reducing costs and increasing accessibility.

排序理由 Introduces a new method for training AI models with a focus on resilience and distributed computing.

在 X — Google DeepMind 阅读 →

AI 生成摘要 · Google Gemini · 来自 6 个来源。 我们如何撰写摘要 →

Google DeepMind unveils Decoupled DiLoCo for resilient AI model training

报道来源 [6]

  1. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    As we push the frontiers of AI infrastructure, our research explores a future where training isn’t constrained by geography, capacity or type of chip.

    As we push the frontiers of AI infrastructure, our research explores a future where training isn’t constrained by geography, capacity or type of chip. Dive into the technical details → https://t.co/tAq2nQ6kTa https://t.co/y49hOiucXf

  2. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    This progress allow us to rethink global compute:

    This progress allow us to rethink global compute: 🔘 We successfully trained a 12B @GoogleGemma model across four US regions using low-bandwidth networks 🔘 We showed we can mix different hardware generations, such as TPU6e and TPUv5p, without slowing down performance during https:…

  3. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    Decoupled DiLoCo is also self-healing.

    Decoupled DiLoCo is also self-healing. We introduced artificial hardware failures during training runs. The system isolated the disruptions and continued operating, while reintegrating offline units when they came back online. https://t.co/DvQsuzbLpW

  4. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    It builds on 2️⃣ earlier advances:

    It builds on 2️⃣ earlier advances: Pathways: an AI system that connects different computer chips, allowing them to share data and work at their own pace. DiLoCo: an approach to minimize the bandwidth needed across distributed centers. Together as Decoupled DiLoCo, it can tackle …

  5. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    Training frontier AI models relies on identical chips staying in near-perfect synchronization. If a single chip fails, the entire training run can stall.

    Training frontier AI models relies on identical chips staying in near-perfect synchronization. If a single chip fails, the entire training run can stall. Decoupled DiLoCo explores how to continuously train AI models without ever stopping due to failures. https://t.co/jbhtWUagBG

  6. X — Google DeepMind TIER_1 English(EN) · GoogleDeepMind ·

    This is Decoupled DiLoCo: our new resilient and flexible way to train advanced AI models across multiple data centres. 🧵 https://t.co/YRmPrqIbYE

    This is Decoupled DiLoCo: our new resilient and flexible way to train advanced AI models across multiple data centres. 🧵 https://t.co/YRmPrqIbYE

相关实体

相关话题