Release Gateway-v0.3.1
SGLang has released version 0.3.1 of its model gateway, significantly boosting performance and reducing memory usage. The update introduces cache-aware routing that is 10-12x faster and uses 99% less memory, enabling 100x more cache entries within the same footprint. This release also incorporates enterprise-grade security features like JWT/OIDC authentication and adds support for classification workloads. AI
IMPACT Enhances efficiency and scalability for large-scale multi-tenant AI deployments.