Next-Token Prediction Learns Generalisable Representations of Sleep Physiology
Researchers have developed Hypnos, a new foundation model for sleep physiology that utilizes next-token prediction for representation learning. Trained on eight different sensing modalities from over 20,000 polysomnography recordings, Hypnos tokenizes physiological signals and uses an auto-regressive RQ-Transformer to predict future data points. This approach significantly outperforms existing models on various benchmarks, including sleep stage classification and atrial fibrillation detection, while requiring substantially less labeled data. AI
IMPACT Demonstrates a novel self-supervised learning approach for multi-modal physiological data, potentially improving healthcare diagnostics with less labeled data.