Flex-MemoryLLM

Apple unveils MemoryLLM for interpretable Transformer FFNs

Apple's Machine Learning Research team has introduced MemoryLLM, a novel approach to enhance the interpretability of feed-forward networks (FFNs) within Transformer models. By decoupling FFNs from self-attention mechani…