Brief

A new research paper investigates how the allocation of sparsity in neural networks impacts their ability to recover accuracy after pruning, especially when labeled retraining data is unavailable. The study compares different sparsity allocation methods like ERK and LAMP across various datasets and architectures, finding that the choice of allocation significantly affects post-repair accuracy. Researchers identified a critical transition regime where standard repair methods begin to fail, highlighting the need to jointly consider pruning allocation and repair strategies. AI

Researchers have developed a new method called Early High-Frequency Injection (EIHF) to improve out-of-distribution (OOD) detection in computer vision models. EIHF works by injecting high-frequency information into the input data before it's processed by the first convolution layer, without altering the training objective. This approach enhances the model's ability to distinguish between in-distribution and out-of-distribution data, particularly for geometry-sensitive tasks, by reshaping feature geometry and reducing overlap in scores. Experiments on CIFAR-100 and ImageNet-100 datasets showed promising results, including improved false positive rates and area under the receiver operating characteristic curve. AI

Researchers have introduced HamJEPA, a novel approach to Joint Embedding Predictive Architectures (JEPAs) that moves beyond isotropic regularization. This new method encodes views as phase-space states and uses a learned Hamiltonian leapfrog map for cross-view prediction. Experiments on CIFAR-100 and ImageNet-100 show significant improvements in kNN and linear probe accuracy compared to existing methods like SIGReg. AI