Lei Gao

DuetServe framework optimizes LLM serving with adaptive GPU multiplexing

Researchers have developed DuetServe, a new framework designed to optimize the serving of large language models (LLMs). This system addresses the challenge of balancing high throughput with low latency by intelligently …