nvlabs.github.io

Sana system optimizes LLM performance with dynamic resource allocation

Researchers have introduced Sana, a novel system designed to enhance the efficiency of large language models (LLMs) by dynamically allocating computational resources. Sana achieves this by routing queries to either a fa…