New framework enables on-chip fine-tuning for photonic vision transformers

Researchers have developed Opto-ViT-v2, a novel framework enabling parameter-efficient fine-tuning of vision transformers directly on photonic accelerators. This system addresses challenges in on-chip training by reducing activation storage and weight updates through tensorized low-rank decomposition. Opto-ViT-v2 also incorporates a noise model calibrated with real-world device measurements, demonstrating robustness to photonic noise and achieving high energy efficiency for edge vision systems. AI

IMPACT Enables more efficient and localized AI model adaptation on edge devices with photonic hardware.

RANK_REASON The cluster contains an academic paper detailing a new technical framework for specialized hardware. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• FGVC
• Opto-ViT-v2
• parameter-efficient fine-tuning
• vision transformer
• VTAB-1K

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →