BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting
Researchers have developed BEAST3D, a self-supervised framework designed to learn 3D visual representations from unlabeled, multi-view animal video recordings. This method utilizes a vision transformer to predict 3D Gaussian splats, enabling reconstruction of novel views and segmentation of animals from backgrounds. BEAST3D effectively reconstructs 3D structure even with as few as four camera views, making it suitable for specialized laboratory settings. The framework's learned features have demonstrated strong performance in downstream tasks including novel view synthesis, multi-view pose estimation for behavioral analysis, and neural encoding. AI
IMPACT Enables more detailed and automated analysis of animal behavior from existing video data, potentially accelerating neuroscience research.