FrequencyFormer pipeline boosts vision transformer efficiency for edge devices

Researchers have developed FrequencyFormer, a novel pipeline designed to make vision transformers (ViTs) more efficient for deployment on sensor-edge systems. This approach leverages the frequency domain to compress image data, reducing the energy and bandwidth needed for transmission from sensors to processors. FrequencyFormer utilizes a multi-scale DCT tokenizer to create compact frequency-domain tokens, achieving significant data reduction with minimal accuracy loss. It also incorporates a near-sensor hardware implementation and a modified communication architecture to further enhance energy efficiency, demonstrating a substantial improvement in performance metrics like TOPS/W and communication energy. AI

IMPACT Enables more efficient deployment of vision transformers on resource-constrained edge devices.

RANK_REASON The cluster contains a research paper detailing a new technical approach. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.CV →

• discrete cosine transform
• FrequencyFormer
• MIPI Alliance
• vision transformer
• Vít

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →