DeltaNet
PulseAugur coverage of DeltaNet — every cluster mentioning DeltaNet across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Research identifies parasitic pathway hindering RNN state tracking
A new research paper introduces the concept of an "additive input pathway" in Householder linear RNNs, identifying it as a parasitic attractor that hinders state tracking. When this pathway is removed, the same architec…
-
Nexus model offers efficient text-to-image generation comparable to SDXL
Researchers have introduced Nexus, a novel text-to-image generation model designed for enhanced efficiency. Nexus integrates a sparse architecture, linear complexity, and low-bit quantization, combining MoE feed-forward…
-
Qwen3.8-27B-FP8 multimodal model released with Gated-DeltaNet architecture
A new multimodal model named Qwen3.8-27B-FP8 has been released. This model is capable of image-text-to-text generation and features a hybrid Gated-DeltaNet/Gated-Attention architecture. It includes 64 layers, 27 billion…
-
New TTCD framework enhances long-context language modeling during inference
Researchers have introduced Test-Time Context Distillation (TTCD), a novel framework for long-context language modeling that optimizes parameter updates during inference. Unlike previous methods, TTCD incorporates a sel…
-
Kimi Delta Attention Explained: From Quadratic to Linear Variants
This article delves into the Kimi Delta Attention (KDA) mechanism, a sophisticated variant of linear attention. It traces the evolution from quadratic attention to KDA, explaining how KDA addresses the limitations of ea…
-
Reverso: Efficient Time Series Foundation Models for Zero-Shot Forecasting
Researchers have developed Reverso, a new family of efficient time series foundation models designed for zero-shot forecasting. Unlike previous models that relied on large-scale transformers and hundreds of millions of …
-
New research explores unified routing for adaptive LLM efficiency · 2 sources tracked
Two new research papers explore methods to optimize the efficiency of large language models by dynamically adjusting computational resources based on token complexity. The first paper, "Linear Attention Architectures," …
-
New decoder architecture shows improved retention in small code models
Researchers have developed SamatNext v0.2-B, a 356M-parameter hybrid sequence decoder designed to mitigate forgetting in small code models during sequential fine-tuning. This experimental model alternates Differential-A…
-
New framework unifies sequence models using Bayesian memory
Researchers have introduced a "design-model" framework for creating efficient recurrent sequence maps based on memory assumptions. This framework uses Bayesian filtering to write evidence into memory and a query-depende…
-
New attention methods tackle LLM long-context challenges
Researchers are developing new attention mechanisms to handle increasingly long contexts in large language models. One approach, Runtime-Certified Bounded-Error Quantized Attention, uses tiered KV caches to compress mem…