Lngram v2

Lngram v2 enhances transformer memory with interpretable discrete representations

Researchers have introduced Lngram v2, an advancement in latent n-gram memory designed for transformers. This new version decouples memory capacity from backbone width, allowing for independent scaling and reduced compu…