enwik8
PulseAugur coverage of enwik8 — every cluster mentioning enwik8 across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
New Frustrated Synchronization Network challenges transformer performance
Researchers have introduced the Frustrated Synchronization Network (FSN), a novel attention architecture inspired by the synchronization of oscillators. Unlike traditional attention mechanisms, the FSN's computation is …
-
New ELM Network Mimics Cortical Neurons, Improves Sequence Modeling
Researchers have introduced the Expressive Leaky Memory (ELM) Network, a novel recurrent neural network architecture designed to better mimic the functional components of cortical neurons. This new model allows for inde…
-
Learned token routing in transformers adapts computation depth for efficiency
Researchers have developed a new technique called Token-Selective Attention (TSA) for transformer models that allows them to dynamically adjust the computation depth for each token. This method uses a lightweight, learn…
-
StateSMix compressor uses Mamba SSMs and n-grams for online lossless compression
Researchers have developed StateSMix, a novel lossless compression algorithm that utilizes Mamba-style State Space Models (SSMs) combined with sparse n-gram context mixing. This system trains token-by-token on the data …