Google DeepMind 推出了 Decoupled DiLoCo，这是一种新颖的先进 AI 模型训练方法，可增强跨数据中心的弹性和灵活性。该系统可以在使用低带宽网络的地理分散区域跨多个数据中心训练 Google 的 12B Gemma 模型，甚至可以混合不同代硬件，例如 TPU6e 和 TPUv5p。Decoupled DiLoCo 被设计为自愈的，能够隔离并继续训练，即使出现人工硬件故障，并在单元恢复联机时重新集成，从而解决了通常会阻碍 AI 训练的同步问题。 AI
影响 能够实现更强大、更灵活的大规模 AI 模型训练，可能降低成本并提高可访问性。
排序理由 引入了一种新的 AI 模型训练方法，侧重于弹性和分布式计算。
AI 生成摘要 · Google Gemini · 来自 6 个来源。 我们如何撰写摘要 →
As we push the frontiers of AI infrastructure, our research explores a future where training isn’t constrained by geography, capacity or type of chip. Dive into the technical details → https://t.co/tAq2nQ6kTa https://t.co/y49hOiucXf
This progress allow us to rethink global compute: 🔘 We successfully trained a 12B @GoogleGemma model across four US regions using low-bandwidth networks 🔘 We showed we can mix different hardware generations, such as TPU6e and TPUv5p, without slowing down performance during https:…
Decoupled DiLoCo is also self-healing. We introduced artificial hardware failures during training runs. The system isolated the disruptions and continued operating, while reintegrating offline units when they came back online. https://t.co/DvQsuzbLpW
It builds on 2️⃣ earlier advances: Pathways: an AI system that connects different computer chips, allowing them to share data and work at their own pace. DiLoCo: an approach to minimize the bandwidth needed across distributed centers. Together as Decoupled DiLoCo, it can tackle …
Training frontier AI models relies on identical chips staying in near-perfect synchronization. If a single chip fails, the entire training run can stall. Decoupled DiLoCo explores how to continuously train AI models without ever stopping due to failures. https://t.co/jbhtWUagBG
This is Decoupled DiLoCo: our new resilient and flexible way to train advanced AI models across multiple data centres. 🧵 https://t.co/YRmPrqIbYE