cross entropy
PulseAugur coverage of cross entropy — every cluster mentioning cross entropy across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
New Ordinal Cross-Entropy framework enhances deep learning for medical predictions
Researchers have introduced a new framework called Ordinal Cross-Entropy (OCE) designed to improve the accuracy of deep neural networks in medical applications where target labels have an inherent ordinal structure. Tra…
-
Mean Field Control Analysis of Transformer Layers under Cross-Entropy Training
Researchers have analyzed Transformer layers within a cross-entropy training framework using a continuous-depth mean field control perspective. They treat depth as time and layer parameters as controls, modeling the Tra…
-
New research probes Transformer energy use, learned linearity, and training dynamics
Recent research explores the intricacies of Transformer models, focusing on their energy consumption, internal linear properties, and training dynamics. One paper introduces a scaling model to predict energy usage durin…
-
Weight norm's role in neural network grokking clarified
Researchers have investigated the phenomenon of 'grokking' in neural networks, where a model transitions from memorization to generalization. Their findings indicate that the weight norm, previously thought to be the pr…
-
New losses achieve Neural Collapse faster in supervised learning
Researchers have introduced new methods, NTCE and NONL, to improve supervised classification by achieving Neural Collapse (NC) more efficiently. These techniques address limitations in existing paradigms like cross-entr…
-
Researchers develop Evolutionary Dynamic Loss for distribution-free pretraining
Researchers have developed a new framework called Evolutionary Dynamic Loss (EDL) for pretraining classification losses. EDL learns a transferable loss function using synthetic data, avoiding the need for real samples d…
-
Linear-Core Surrogates offer smooth loss functions with linear rates for classification
Researchers have introduced Linear-Core (LC) Surrogates, a novel family of convex loss functions designed to combine the benefits of smooth and piecewise-linear losses in machine learning. These surrogates are different…
-
Contrastive learning advances model robustness and transparency in AI
Contrastive learning is a machine learning technique that creates an embedding space where similar data points are grouped together and dissimilar ones are separated. This method can be applied in both supervised and un…