Researchers have adapted a self-supervised learning architecture, a joint embedding predictive architecture (JEPA), to decode sensory-evoked functional magnetic resonance imaging (fMRI) data. This approach trains encoders on brain data alone, without direct access to stimulus images. Applied to the Natural Scenes Dataset (NSD), the learned brain embeddings show competitive performance on downstream tasks like brain captioning and image reconstruction, even outperforming some supervised methods. Notably, encoders trained independently for different participants can be aligned unsupervised, allowing a decoding head trained on one individual to be applied to another with minimal performance loss. AI
IMPACT This research could enable pooling of fMRI data across participants and datasets by removing the need for direct stimulus access during training.
RANK_REASON The item is an academic paper detailing a novel methodology for analyzing fMRI data using self-supervised learning. [lever_c_demoted from research: ic=1 ai=1.0]
- alphaXiv
- arXiv
- cs.LG
- DagsHub
- functional magnetic resonance imaging
- Hugging Face
- IArxiv
- Natural Scenes Dataset (NSD)
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
