SIGReg
PulseAugur coverage of SIGReg — every cluster mentioning SIGReg across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
LeJEPA architecture adapted for molecular graph encoders
Researchers have adapted LeJEPA, a self-supervised pretraining architecture, for molecular graph neural networks to assess its impact on molecular property prediction. While pretraining enhances learned representations,…
-
New method tackles "physical representation laziness" in AI world models
Researchers have identified a new failure mode in latent world models called "physical representation laziness," where learned latent states fail to capture crucial physical properties, leading to planning errors. To ad…
-
Polis framework advances 3D self-supervision for city-scale environments
Researchers have introduced Polis, a novel self-supervised learning framework designed for large-scale 3D urban environments. Unlike existing models trained on indoor or object-level data, Polis utilizes a mixture of 12…
-
New SciJEPA framework advances scientific document representation
Researchers have developed SciJEPA, a new framework for learning scientific document representations. This citation-free approach uses asymmetric within-document prediction, where title and abstract representations pred…
-
Vision Transformer encoders can grow dynamically to match task complexity
Researchers have introduced Successive Capacity Growth (SCG), a novel method for expanding Vision Transformer encoders in Joint-Embedding Predictive Architectures (JEPAs) for world modeling. SCG begins with a minimal en…
-
New video pretraining method and 10M-hour dataset released
Researchers have introduced LeVJEPA, a novel video pretraining method that significantly reduces computational costs while maintaining or improving downstream accuracy. This approach bypasses common heuristics like arch…
-
New JEPA method uses contrastive inverse dynamics to improve world models
Researchers have developed a new method called Action-Contrastive Masked Transition Modeling (AC-MTM) for Joint-Embedding Predictive Architectures (JEPAs) that addresses the issue of trivial solutions in world models. U…
-
New SCALE method enhances AI planning by improving latent space geometry
Researchers have developed SCALE (State-Calibrated Latent Embeddings), a new method to improve planning in joint-embedding predictive world models. SCALE enhances the geometric properties of latent representations, simi…
-
New SIGReg Method Boosts Multi-Task World Model Learning
Researchers have developed a new method called Temporally Centered SIGReg to improve multi-task learning in world models. The original SIGReg technique, while effective for single tasks, struggles with multiple tasks by…
-
Temporal-Distance JEPA enhances world model predictive control
Researchers have introduced Temporal-Distance JEPA (TD-JEPA), a novel approach to representation learning for latent world model predictive control. This method enhances Joint-Embedding Predictive Architectures (JEPAs) …
-
New JEPA World Models Improve Robot Data Transferability with Depth Prior
Researchers have developed a new method for training world models, particularly those based on the Joint Embedding Predictive Architecture (JEPA), to improve their ability to learn from complex real-world robot data. By…
-
New training method enhances robot navigation with latent imagination
Researchers have developed a new method for training navigation policies in legged robots that enhances their ability to anticipate and react to dynamic environments. By incorporating a lightweight predictive supervisio…
-
New techniques aim to improve LLM KV-cache efficiency and accuracy
Researchers are exploring novel methods to improve the efficiency of large language models by optimizing their KV cache, a component crucial for inference but known for its high memory and bandwidth demands. One approac…
-
AV-JEPA model advances audio-visual self-supervised learning
Researchers have introduced AV-JEPA, a new self-supervised learning model that extends LeJEPA to handle both audio and visual data. This model utilizes an early-fusion Vision Transformer and modality dropout for masking…
-
New theory links JEPA world models to Active Inference via SIGReg objective
A new theoretical paper proposes that the SIGReg objective, when used as an anti-collapse regularizer in Joint-Embedding Predictive Architectures (JEPAs), can serve as a valid Active Inference (AIF) variational free ene…
-
New Kepler-Encoder-v0.1 model fuses robot state with vision
Researchers have developed Kepler-Encoder-v0.1, a novel multimodal embedding model designed for robots. This model integrates visual data with proprioception and force/torque sensor information into a unified latent spa…
-
New research shows local synaptic rules can enable self-supervised learning without backpropagation
Researchers have demonstrated that local synaptic learning rules, specifically spike-timing-dependent plasticity (STDP+) and homeostatic plasticity, can effectively implement a SIGReg gradient for self-supervised learni…
-
LeNEPA: New Time-Series SSL Method Reduces Reliance on Data Augmentation
Researchers have introduced LeNEPA, a novel self-supervised learning method for time-series data that does not require data augmentation. LeNEPA utilizes a causal backbone and a next-latent-token prediction objective, e…
-
New statistical regularizers enhance self-supervised learning stability
Researchers have introduced a new family of statistical regularizers for Self-Supervised Learning (SSL) that aim to improve representation collapse prevention. The proposed methods analytically integrate random projecti…
-
Yann LeCun develops highly efficient AI model trainable on single GPU
Yann LeCun is developing a novel AI model architecture designed for extreme efficiency. This new model boasts a mere 15 million parameters, allowing it to be trained on a single GPU in just a few hours. The approach inc…