NuWa: Deriving Lightweight Class-Specific Vision Transformers for Edge Devices
Researchers have developed NuWa, a novel method for creating lightweight, class-specific Vision Transformers (ViTs) optimized for edge devices. Existing compression techniques often retain redundant information, leading to suboptimal performance on specialized tasks. NuWa addresses this by purifying knowledge to remove class-detrimental weights and using closed-form optimization to efficiently derive compact ViTs. This approach significantly speeds up inference and improves accuracy for specific classes without requiring post-pruning retraining, outperforming current methods in both efficiency and performance. AI
IMPACT Enables more efficient deployment of advanced vision models on resource-constrained edge devices.