研究人员开发了 Strait，一个旨在提高机器学习推理服务效率的新系统，特别是在本地环境中。Strait 通过模拟潜在的争用和内核执行干扰，解决了任务优先级排序和延迟估算方面的局限性。这种感知优先级的调度旨在提高在 GPU 利用率高的情况下，高优先级推理任务的截止日期满足率，并显著减少截止日期违规情况。 AI
影响 提高了机器学习推理服务的效率和截止日期遵守率，可能支持更强大的本地部署。
排序理由 描述机器学习推理服务新系统的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2604.28175v1 Announce Type: new Abstract: Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. However, limited support for task prioritization and insufficient latency estimatio…
Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. However, limited support for task prioritization and insufficient latency estimation under concurrent execution may restrict their …
Machine learning (ML) inference serving systems host deep neural network (DNN) models and schedule incoming inference requests across deployed GPUs. However, limited support for task prioritization and insufficient latency estimation under concurrent execution may restrict their …