New video transformer mimics primate vision and brain activity

Researchers have developed a novel video transformer architecture inspired by primate vision and neural data. This model incorporates a sparse token selection module to enhance efficiency and mimic biological visual processing. It also features a split-and-fuse design with distinct "what" and "where" streams, which are combined before classification. The model demonstrates competitive accuracy and inference time on standard benchmarks and shows improved brain-model correlation with EEG data, suggesting its effectiveness in brain-aligned video understanding. AI

IMPACT This research could lead to more interpretable and efficient video understanding models by incorporating biological principles.

RANK_REASON The item is a research paper published on arXiv detailing a new model architecture. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• alphaXiv
• AmirHosein Fadaei
• arXiv
• CatalyzeX Code Finder for Papers
• CORE Recommender
• DagsHub
• electroencephalography
• Gotit.pub
• Hugging Face
• Influence Flower
• Kinetics-400
• ScienceCast
• Something-Something V2

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →