Flash EQ-Linear accelerates equivariant neural networks with Fourier transforms

Researchers have developed Flash EQ-Linear, a novel algorithm designed to significantly accelerate equivariant linear layers in neural networks. By leveraging the Fourier convolution theorem and the conjugate symmetry of the real Discrete Fourier Transform, this method reduces computational complexity. Dedicated CUDA kernels have been implemented for both forward and backward passes, achieving up to a 2x speedup at the operator level and up to a 1.7x end-to-end speedup for networks like Flash EQ-ViT and Flash EQ-Swin compared to existing equivariant and non-equivariant baselines. AI

IMPACT This development offers significant speedups for equivariant neural networks, potentially making them more practical for a wider range of vision tasks.

RANK_REASON The cluster describes a new algorithm and its implementation for accelerating a specific type of neural network layer, detailed in an arXiv paper. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• CUDA
• density functional theory
• Equivariant Networks for Hierarchical Structures
• Flash EQ-Linear
• Flash EQ-Swin
• Flash EQ-ViT
• Fourier convolution theorem
• PyTorch

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →