两篇新研究论文探讨了高效运行大型语言模型（LLM）的挑战。第一篇论文研究了在智能手机和专用NPU等边缘设备上部署LLM的性能权衡，强调了热限制和内存带宽限制。第二篇论文介绍了一个使用启发式算法的可扩展框架，用于优化异构GPU云环境中LLM推理的资源分配，旨在满足服务水平目标的同时最大限度地降低成本。 AI
影响 这些论文为优化LLM在设备端和云端的性能和成本提供了见解，这对于扩展AI应用至关重要。
排序理由 该集群包含两篇讨论LLM推理性能和资源分配的学术论文。
AI 生成摘要 · Google Gemini · 来自 3 个来源。 我们如何撰写摘要 →
arXiv:2505.04021v3 Announce Type: replace-cross Abstract: Inference providers must maintain availability for many LLMs, including low-volume but essential models, making resource efficiency increasingly important as token prices fall. Analysis of production traces reveals a dynam…
arXiv:2603.23640v2 Announce Type: replace-cross Abstract: Deploying large language models on-device for always-on personal agents demands sustained inference from hardware tightly constrained in power, thermal envelope, and memory. We benchmark Qwen 2.5 1.5B (4-bit quantised) acr…
arXiv:2604.07472v2 Announce Type: replace Abstract: Serving large language model (LLM) inference in cloud environments requires jointly optimizing model selection, GPU provisioning, parallelism configuration, and workload routing under latency, accuracy, memory, and budget constr…