FSDP2
PulseAugur coverage of FSDP2 — every cluster mentioning FSDP2 across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
Unsloth releases DeepSeek-V4.1 and FLUX.3 with enhanced features
Unsloth has released updates for DeepSeek-V4.1 and FLUX.3, enabling local execution of these models. The DeepSeek-V4.1-Flash-1017 model boasts 552 billion parameters with 16 billion active and a 1 million token context …
-
Databricks cuts Qwen LLM fine-tuning time by 6x with AI Runtime optimizations
Databricks has detailed how they achieved a sixfold speedup in fine-tuning the Qwen 3.8-27B large language model using their AI Runtime. By implementing specific kernel installations and optimizing the data pipeline, th…
-
LLM efficiency, scaling, and deployment strategies detailed across multiple sources
A recent paper benchmarks the energy efficiency of locally deployed large language models (LLMs) on consumer hardware, finding that factors beyond parameter count, such as model architecture and quantization, significan…
-
User seeks automatic batch size detection for Accelerate + FSDP2 multi-GPU training
A user on Reddit's r/MachineLearning subreddit is seeking guidance on how to automatically determine an optimal batch size for multi-GPU training using Accelerate with FSDP2. They are looking for a feature similar to Hu…
-
New methods tackle memory peaks for long-context MoE training
Researchers have developed four novel techniques to address memory limitations in training Mixture-of-Experts (MoE) models with long contexts. These methods, PipelinedLLEP, Ring-DTP, Selective Checkpoint Offload (SCO), …
-
NVIDIA and Hugging Face launch NeMo Automodel for scalable diffusion model fine-tuning
NVIDIA and Hugging Face have collaborated to release NeMo Automodel, an open-source PyTorch library designed to streamline the fine-tuning of diffusion models. This new tool allows users to train and fine-tune models in…