DeltaNet
PulseAugur coverage of DeltaNet — every cluster mentioning DeltaNet across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
LLM memory challenge: Updating outdated info, not just recall
The current focus on increasing the context window size of large language models may be overlooking a critical aspect of memory: the ability to update or forget outdated information. While models can now process million…
-
KronLSTM introduces Kronecker Delta Rule for stable long-context AI memory
Researchers have introduced KronLSTM, a novel linear attention architecture that utilizes a Kronecker Delta Rule for stable associative memory updates. This approach enhances long-context modeling by allowing for precis…
-
AI autonomously discovers 105 neural architectures, outperforming Mamba2
Researchers have developed ASI-Arch, an AI system designed to autonomously conduct research into neural architectures. This system operates through a continuous cycle of research, experimentation, analysis, and updates.…
-
New research tackles recurrent neural network memory and efficiency
Two new research papers explore advancements in recurrent neural networks for handling sequential data. The first paper, "DeltaTTT," introduces a layerwise optimization technique for nonlinear recurrent memory networks,…
-
New Retrieval-Centric Deep Learning Paradigm for Growing Neural Networks
Researchers have introduced Retrieval-Centric Deep Learning (RCDL), a novel paradigm for neural networks that stores key-value representations for each training data point. Unlike traditional methods that compress data …
-
Spiking neural networks advance energy-efficient language modeling · 3 sources tracked
Researchers are exploring the development of spiking neural networks (SNNs) for language modeling, aiming for greater energy efficiency compared to traditional transformers. One approach, NeuronSpark, introduces adaptiv…
-
LLMs guide evolutionary search for native spiking neural network architectures
Researchers have developed OpenArchEvo, a novel system that leverages large language models (LLMs) to evolve native neural architectures for spiking neural networks (SNNs). Unlike traditional methods that adapt existing…
-
New research advances tabular foundation models for efficiency and robustness
Multiple research papers explore advancements in tabular foundation models (TFMs), focusing on improving their efficiency, robustness, and adaptability. Techniques such as knowledge distillation are being used to create…
-
New research tackles catastrophic forgetting in continual learning models · 8 sources tracked
Multiple research papers explore the complex dynamics of continual learning in neural networks, focusing on how models retain or forget information over time. Studies investigate methods to mitigate catastrophic forgett…
-
Local 27B LLM audits and fixes Splunk + Sysmon stack
A security analyst used a local 27B LLM, Qwen3.8-27B, to audit and fix their Splunk and Sysmon security stack. The model, running entirely on the analyst's hardware, successfully identified and corrected issues such as …
-
Research identifies parasitic pathway hindering RNN state tracking
A new research paper introduces the concept of an "additive input pathway" in Householder linear RNNs, identifying it as a parasitic attractor that hinders state tracking. When this pathway is removed, the same architec…
-
Nexus model offers efficient text-to-image generation comparable to SDXL
Researchers have introduced Nexus, a novel text-to-image generation model designed for enhanced efficiency. Nexus integrates a sparse architecture, linear complexity, and low-bit quantization, combining MoE feed-forward…
-
Qwen3.8-27B-FP8 multimodal model released with Gated-DeltaNet architecture
A new multimodal model named Qwen3.8-27B-FP8 has been released. This model is capable of image-text-to-text generation and features a hybrid Gated-DeltaNet/Gated-Attention architecture. It includes 64 layers, 27 billion…
-
New TTCD framework enhances long-context language modeling during inference
Researchers have introduced Test-Time Context Distillation (TTCD), a novel framework for long-context language modeling that optimizes parameter updates during inference. Unlike previous methods, TTCD incorporates a sel…
-
Kimi Delta Attention Explained: From Quadratic to Linear Variants
This article delves into the Kimi Delta Attention (KDA) mechanism, a sophisticated variant of linear attention. It traces the evolution from quadratic attention to KDA, explaining how KDA addresses the limitations of ea…
-
Reverso: Efficient Time Series Foundation Models for Zero-Shot Forecasting
Researchers have developed Reverso, a new family of efficient time series foundation models designed for zero-shot forecasting. Unlike previous models that relied on large-scale transformers and hundreds of millions of …
-
New research explores unified routing for adaptive LLM efficiency · 2 sources tracked
Two new research papers explore methods to optimize the efficiency of large language models by dynamically adjusting computational resources based on token complexity. The first paper, "Linear Attention Architectures," …
-
New decoder architecture shows improved retention in small code models
Researchers have developed SamatNext v0.2-B, a 356M-parameter hybrid sequence decoder designed to mitigate forgetting in small code models during sequential fine-tuning. This experimental model alternates Differential-A…
-
New framework unifies sequence models using Bayesian memory
Researchers have introduced a "design-model" framework for creating efficient recurrent sequence maps based on memory assumptions. This framework uses Bayesian filtering to write evidence into memory and a query-depende…
-
New attention methods tackle LLM long-context challenges
Researchers are developing new attention mechanisms to handle increasingly long contexts in large language models. One approach, Runtime-Certified Bounded-Error Quantized Attention, uses tiered KV caches to compress mem…