Ternary transformers

New method enables efficient fine-tuning of ternary transformers

Researchers have developed a new method called ternary multiplicative adaptation for fine-tuning transformers that are quantized to ternary weights. This approach uses a low-rank Kronecker factorization to represent dis…