ZeRO-3
PulseAugur coverage of ZeRO-3 — every cluster mentioning ZeRO-3 across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New repo enables scaled self-distillation with EMA teacher using ZeRO-3 and LoRA
A new repository has been released that enables self-distillation with an Exponential Moving Average (EMA) teacher to function at scale, even with large models. This method utilizes LoRA finetuning, sharing a frozen bac…
-
NVIDIA B300 fine-tuning of Qwen3-32B detailed in new research
A new paper details the operational challenges and solutions encountered when fine-tuning the Qwen3-32B model on NVIDIA's B300 accelerators. The research focuses on practical aspects of multi-node training, offering ins…
-
Open-source C++/CUDA infra trains trillion-parameter LLMs
A developer has created TitanCore Core-1, an open-source infrastructure for training trillion-parameter LLMs. Written in C++ and CUDA, it targets VRAM limitations by implementing ZeRO-3 FSDP and fused kernels. This appr…