DistilGPT-2
PulseAugur coverage of DistilGPT-2 — every cluster mentioning DistilGPT-2 across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Stochastic Rounding Improves Low-Precision Transformer Inference
Researchers have investigated the use of stochastic rounding (SR) versus round-to-nearest (RN) for low-precision transformer inference, finding that the optimal method depends on the specific layer within the model. The…
-
AI model weights show evolving local linear structures
Researchers have investigated the nature of linear structures within neural network weights and activations, finding that while local low-rank structures exist, they are not stationary. The study, conducted on synthetic…
-
Model multiplicity defends small language models against edge device attacks
Researchers have developed a novel defense system called "model multiplicity" to detect adversarial attacks during the training of small language models on edge devices. This approach involves training multiple language…