Bank of Values (BoV)

New LLM architecture decouples value vectors from residual stream

Researchers have explored a novel approach to transformer architecture in large language models, suggesting that value vectors in deeper layers may not require context from the residual stream. Their findings indicate t…