Megatron-Core
PulseAugur coverage of Megatron-Core — every cluster mentioning Megatron-Core across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
FlashMask-3 accelerates LLM training with optimized attention kernel
Researchers have developed FlashMask-3, a novel sparse-mask kernel designed to accelerate long-context attention in large language and multimodal models. This new kernel optimizes attention mechanisms for Hopper archite…
-
Hugging Face releases Olmo-core 3 for scalable trillion-parameter MoE training
Hugging Face has released Olmo-core 3, an open-source framework designed to scale the training of Mixture-of-Experts (MoE) large language models to trillion-parameter sizes. This new system addresses the computational i…
-
New SCOUT framework detects failures in LLM pre-training
Researchers have developed SCOUT, a new framework designed to pinpoint failures during large language model (LLM) pre-training. SCOUT operates by establishing a strict majority consensus among equivalent replicas to ide…