Brief

Researchers have demonstrated that subliminal learning in neural networks, where knowledge is transferred via task-unrelated data, is primarily governed by compatible output heads rather than shared model initialization. By splitting outputs into auxiliary and class heads, they showed that compatible auxiliary heads facilitate the transfer of teacher signals, improving student model representations. This mechanism allows students trained on noise to achieve performance comparable to teachers, providing a theoretically grounded understanding of subliminal learning and its limitations. AI

Researchers have introduced Federated Martingale Posterior (FMP) sampling, a novel protocol for federated Bayesian neural networks. This method addresses the difficulty of specifying priors in large models by using a predictive distribution and refitting. FMP sampling allows clients to upload data embeddings, enabling the server to run the predictive sampler centrally, thus avoiding the need to share local datasets. Experiments on standard datasets demonstrate that FMP closely matches centralized performance and offers improved calibration compared to existing consensus methods. AI

Two recent arXiv preprints explore high-dimensional ridge regression for non-identically distributed data, moving beyond standard assumptions of independent and identically distributed samples. The papers introduce variance profile models to analyze the predictive risk of ridge estimators, particularly focusing on the double descent phenomenon. Researchers used tools from random matrix theory and operator-valued free probability to derive asymptotic equivalents for risk and degrees of freedom, with numerical experiments validating their findings and highlighting how heterogeneous variance profiles can alter generalization behavior. AI

Researchers have developed a new method called EnCAgg to improve the robustness of federated learning against dynamic model poisoning attacks. This approach uses a small set of known benign clients as references to accurately identify and filter out malicious gradients. The method incorporates density-based clustering in a low-dimensional space and a gradient generator model to reconnect sparse benign gradients, ultimately allowing more legitimate data to participate in the aggregation process. AI

Researchers are exploring new methods to improve federated learning, a technique for training models across decentralized data sources while preserving privacy. One approach, "Choose Wisely and Privately," uses mutual information and a Potential Federation Loss to proactively select clients whose data maximizes utility and fairness before training begins. Another study introduces a lightweight geometric signal to detect atypical clients by measuring how their local training diverges from the global model's functional behavior. Additionally, new theoretical work establishes general lower bounds for differentially private federated learning protocols and analyzes the trade-offs between centralized and decentralized federated learning architectures. AI

Two new research papers detail advancements in energy-efficient Spiking Neural Networks (SNNs) implemented on Field-Programmable Gate Arrays (FPGAs). The first paper introduces SPIKER-LL, an FPGA accelerator designed for adaptive local learning in SNNs, achieving high accuracy with minimal energy consumption. The second paper presents an FPGA implementation of Spiking Recurrent Cells, demonstrating a balance between biological plausibility and hardware efficiency, with results showing competitive accuracy and reduced energy usage. AI