Isay Katsman

New STATIC technique boosts LLM generative retrieval efficiency on accelerators

Researchers have developed a new technique called STATIC (Sparse Transition Matrix-Accelerated Trie Index for Constrained Decoding) to improve the efficiency of generative retrieval in large language models. This method…