NeurIPS '22

AI inference demands efficient GPU management to avoid VRAM exhaustion and fragmentation

Managing GPU resources for AI workloads, particularly generative media and LLM inference, presents significant challenges due to their high memory and compute demands. Unlike traditional web applications, these tasks ca…