AI Research Atlas

Gated DeltaNet

NVIDIA et al. · 9 December 2024

Combines Mamba-2-style gating with the delta rule for targeted memory updates, beating Mamba2 and DeltaNet in a linear-attention layer.

Gating erases memory fast; the delta rule overwrites specific key-value associations. The pair became the standard linear-attention layer in 2025 hybrids (Qwen3-Next, Kimi Linear extends it with finer-grained gating).

Date
Monday, 9 December 2024
Lab
NVIDIA et al.
Kind
paper
Access
open weights

Authors Songlin Yang, Jan Kautz, Ali Hatamizadeh; ICLR 2025 (camera-ready v3 2025-03-06). arXiv v1 2024-12-09.

Sources

  1. arxiv.org/abs/2412.06464
  2. arxiv.org/abs/2510.26692

This record was checked against its sources on 6 October 2026. How we check

Related