New MeanField model improves AI inference scheduling on GPUs

Researchers have developed a MeanField surrogate model to address the challenge of scheduling concurrent AI inference workloads on shared GPUs. This new approach predicts model performance based on local configuration and aggregate GPU state, avoiding the combinatorial complexity of traditional surrogate models. Experiments demonstrated high predictive accuracy and scalability, with the MeanField surrogate enabling a genetic algorithm scheduler to efficiently handle a large number of configurations and avoid service-level agreement violations. AI

IMPACT This MeanField surrogate model offers a scalable solution for optimizing AI inference scheduling on shared GPUs, potentially improving resource utilization and reducing latency in complex AI deployments.

RANK_REASON The cluster contains a research paper detailing a new technical approach. [lever_c_demoted from research: ic=1 ai=1.0]

Read on arXiv cs.AI →

• AI inference
• alphaXiv
• arXiv
• CatalyzeX
• DagsHub
• graphics processing unit
• Hugging Face
• MeanField surrogate
• service-level agreement

• paper
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →