Gated DeltaNet
PulseAugur coverage of Gated DeltaNet — every cluster mentioning Gated DeltaNet across labs, papers, and developer communities, ranked by signal.
11 day(s) with sentiment data
-
New 'onóna' technique enhances language model expressiveness with weight sharing
Researchers have introduced "onóna," a novel parameterization technique for language models that enhances output expressiveness while maintaining input-output weight sharing. This method utilizes two embedding tables an…
-
Alibaba launches Qwen4-Max with 2.4T parameters, cutting costs by 62%
Alibaba Group has released Qwen4-Max, a new 2.4-trillion-parameter sparse mixture-of-experts model. This model features native vision capabilities and a one-million-token context window. A key highlight is its reduced o…
-
SinkState optimizes hybrid LLM serving with low-rank state compression
Researchers have developed SinkState, a novel method to optimize the serving of hybrid Large Language Models (LLMs) by efficiently managing recurrent states. This technique leverages the observation that these states of…
-
Quantization comparison noise floor measured for LLMs
Researchers have quantified the impact of factors other than quantization on the decision-making of large language models. When comparing quantized models to their original versions, changes in the serving engine, reque…
-
KV-cache translation enables seamless model handoffs without re-reading context
Researchers have developed a method called KV-cache translation to enable one language model to seamlessly take over a conversation from another without re-reading the entire history. This technique maps the source mode…
-
New 'onóna' technique boosts Transformer language model expressiveness
Researchers have introduced "onóna," a novel two-table parameterization technique for Transformer language models that enhances output expressiveness while maintaining input-output weight sharing. This method, which inv…
-
New SEED method optimizes hybrid LLM memory and compute efficiency
A new research paper introduces SEED, a method designed to optimize the memory and compute efficiency of hybrid language models. SEED addresses the inefficiencies in how these models handle memory by making the prefill …
-
New Systolic Linear Attention layer boosts TPU performance for large models
Researchers have developed Systolic Linear Attention (SLA), a new attention layer optimized for Google TPUs. Unlike existing gated-delta attention layers tuned for GPUs, SLA is designed to leverage the wide systolic mat…
-
New FlyWheel attention kernels boost TPU performance up to 4.14x
Researchers have developed FlyWheel, a new set of attention kernels optimized for Google's Tensor Processing Units (TPUs). These kernels address inefficiencies in current TPU implementations by reformulating computation…
-
Near-zero embeddings stall gated model training; new fixes proposed
A new paper identifies a critical training instability in gated sequence models, particularly those with untied embeddings. This issue, caused by near-zero input embeddings, can lead to extreme gradient amplification, e…
-
AI model integrates RISC-V CPU into weights, runs Doom
An AI engineer has successfully integrated a 32-bit RISC-V CPU directly into the weights of the Qwen 3.5-4B large language model, creating a system dubbed "Qwenputer." This experimental model can execute arithmetic oper…
-
Developer embeds GPU into LLM weights to run CUDA kernels
A developer has created qwenGPU, an experimental model that integrates a SIMT GPU directly into the weights of Qwen3.5-2B. This novel approach allows the model to execute real CUDA kernels within its own forward passes,…
-
Flide engine accelerates LLM agent decision inference with exact state forking
Researchers have introduced Flide, a novel inference engine designed to accelerate decision-making processes in LLM agent systems. Flide decouples the logical structure of decision requests from their physical execution…
-
LLM power capping ineffective during decode, new research finds
A new research paper challenges the effectiveness of power capping on GPUs for Large Language Model (LLM) serving, particularly during the autoregressive decode phase. The study found that power caps have minimal impact…
-
New DAMI technique boosts long-context retrieval in linear attention models
Researchers have introduced Dual-Axis Memory Initialization (DAMI), a novel technique to improve long-context retrieval in linear attention models. DAMI focuses on optimizing the initialization of gate parameters that c…
-
Qwenputer embeds functional RISC-V CPU within Qwen 3.5 language model weights
Researchers have developed Qwenputer, an experimental model that embeds a functional, deterministic RISC-V CPU directly within the weights of the Qwen 3.5-4B language model. This novel approach utilizes the model's own …
-
NoteBank enhances long-context recall in linear-attention models
Researchers have developed a novel technique called NoteBank to enhance long-context recall in language models that use linear attention or state-space architectures. This method acts as a training-free add-on, storing …
-
LinMorph method learns weight transformation for efficient Video DiT linearization
Researchers have developed LinMorph, a novel method for efficiently linearizing pre-trained Video Diffusion Transformers (Video DiTs). Unlike previous approaches that focus on recovering quality after attention replacem…
-
GateSight optimizes LLM memory by predicting KV cache utility
Researchers have developed GateSight, a novel method to optimize the KV cache in hybrid Large Language Models (LLMs). GateSight uses lightweight MLPs to predict the future utility of KV entries based on signals from Gat…
-
ReplaySSM optimizes State Space Models, boosting decoding speed
Researchers have introduced ReplaySSM, a novel method designed to optimize the performance of State Space Models (SSMs) in AI applications. This technique addresses bottlenecks caused by frequent state input/output oper…