Keda

Kubernetes operators enable scale-to-zero for LLM serving

New Kubernetes operators are emerging to address the cost of running large language models, particularly the issue of idle GPUs burning money. Hearth, an alpha-stage operator, allows users to declaratively serve open-so…