MegaFold: Efficient Training of Next-Generation 3D Attention Protein Models on Cross-Platform GPUs
Researchers have developed MegaFold, a new system designed to make training large-scale 3D attention protein models more efficient. This approach addresses the significant computational and memory challenges posed by models like AlphaFold3, which have cubic scaling costs with sequence length. MegaFold incorporates memory-efficient kernels, communication-efficient sharding, fused operators, and an optimized host-device pipeline to improve GPU utilization. Evaluations on NVIDIA H200 and AMD MI250 GPUs demonstrate MegaFold's ability to train with longer sequences and reduce overall execution time. AI
IMPACT Enables more efficient training of complex protein models, potentially accelerating biomolecular discovery.