New MaRK framework dynamically conditions State Space Models

Researchers have introduced MaRK (Markov-adapted Recurrent Kernels), a novel framework for dynamically conditioning State Space Models (SSMs) in iterative generation tasks. Unlike previous methods that modulated inputs or activations, MaRK directly modifies the SSM's core parameters, including recurrence, read-in, read-out, and discretization. This approach allows each diffusion timestep to reshape the model's memory kernel, offering a more integrated form of conditioning. The framework was tested on a 111M-parameter Hydra SSM backbone using three adapter geometries: Hypernet, Chebyshev polynomial, and Discrete Cosine Transform kernels, with the Chebyshev variant achieving the strongest performance. AI

IMPACT Introduces a new method for conditioning State Space Models, potentially improving their efficiency and adaptability in sequence modeling tasks.

RANK_REASON The cluster contains an academic paper detailing a new method for conditioning State Space Models. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.LG →

• Chebyshev polynomial
• Discrete Cosine Transform
• Hydra
• MaRK
• Markov-adapted Recurrent Kernels
• State Space Models
• Syed Ibrahim Omer

• paper
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →