Can a self-supervised model learn good visual representations without ever reconstructing pixels? JEPA, the program from FAIR now continued at AMI Labs, says ye
Yann LeCun argues that current Large Language Models (LLMs) are not on a path to human-level intelligence because they lack the ability to predict consequences or perform search-based reasoning. He advocates for his Joint Embedding Predictive Architectures (JEPA) approach, which focuses on self-supervised learning of world models. JEPA aims to learn representations by predicting missing data embeddings, a method he believes is more promising for achieving general intelligence. AI
IMPACT Yann LeCun's critique of LLMs and promotion of JEPA suggests a potential shift in AI research focus away from pure language models towards world-model-based approaches for achieving AGI.