新研究优化跨不同GPU和硬件的LLM推理

报道来源 [9]

1. arXiv cs.LG TIER_1 English(EN) · Ruokai Yin, Priyadarshini Panda · 2026-08-04 04:00

   Celty：用于高效双稀疏大模型推理的 SpMspV GPU 内核与 SIMT 协同设计

   arXiv:2608.01536v1 Announce Type: cross Abstract: Large Language Models (LLMs) increasingly rely on sparsity to reduce inference cost, but most prior work targets a single sparsity source-either weight or activation-and optimizes for batched multi-user inference. Dual-sparsity, w…

2. arXiv cs.LG TIER_1 English(EN) · Dzmitry Malyshau · 2026-08-04 04:00

   Meganeura：通过 Vulkan 和 Metal 实现便携式 GPU 训练和推理

   arXiv:2608.01563v1 Announce Type: new Abstract: Training and deployed inference often cross export, conversion, and platform-specific runtime boundaries. Meganeura asks whether one compact native compiler can span both phases on consumer GPUs. Its typed static graph, automatic di…

3. Hugging Face Daily Papers TIER_1 English(EN) · 2026-07-29 07:50

   从 Token 到瓦时：现代 GPU 上 LLM 推理的分析性能耗估算

   The operational energy consumption of large language model (LLM) inference is becoming an increasingly important component of the environmental footprint of deployed AI systems. However, direct measurement of inference energy often requires hardware telemetry, power instrumentati…

4. Hacker News — AI stories ≥50 points TIER_1 English(EN) · Anon84 · 2026-08-03 11:15

   AirLLM 70B 在单块 4GB GPU 上进行推理

5. dev.to — LLM tag TIER_1 English(EN) · Mingxin Technology · 2026-08-05 12:30

   国内加速卡长上下文推理性能评估

   <p>Long-context inference is becoming a core deployment scenario for large language models, and the storage and memory access bottleneck of the KV Cache directly determines the throughput and time-to-first-token (TTFT) of inference systems. The Mingxin FX100, as a domestic storag…

6. r/LocalLLaMA TIER_1 English(EN) · /u/HomoAgens1 · 2026-08-02 00:05

   多个GPU在LLM推理中的扩展性如何？（试图理解基础知识）

   <!-- SC_OFF --><div class="md"><p>Hi everyone,<br /> I’m fairly new to the multi-GPU side of local LLMs and I’m trying to understand how inference actually scales across multiple GPUs.</p> <p>Suppose I have a model running on a single GPU and then move to two or more GPUs using l…

7. Mastodon — mastodon.social TIER_1 English(EN) · [email protected] · 2026-08-03 14:30

   Big Walk 就像……《无名鹅戏》很难超越。那是一次愚蠢的体验，捕捉了我所想象的成为一只感性鹅的感觉

   Big Walk is like… Untitled Goose Game is a tough act to follow. It was a silly experience that captured what I imagine it would feel like to be a sentient goose: a lot of waddling, a lot of honking, and a lot of shenanigans. That's why B… https://www. theverge.com/games/973166/bi…

   链接 theverge.com/…/big-walk-review
8. Mastodon — mastodon.social TIER_1 English(EN) · [email protected] · 2026-08-03 14:27

   AirLLM 70B 在单块 4GB GPU 上进行推理 文章链接: https:// github.com/lyogavin/airllm 评论链接: https:// news.ycombinator.com/item?id=4 9154228 点数:

   AirLLM 70B inference with single 4GB GPU Article URL: https:// github.com/lyogavin/airllm Comments URL: https:// news.ycombinator.com/item?id=4 9154228 Points: 6 # Comments: 1 https:// github.com/lyogavin/airllm # Tech # Technology # TechNews # AI # Gadgets # Software # Cybersecu…

   链接 github.com/…/airllm
9. Mastodon — mastodon.social TIER_1 English(EN) · [email protected] · 2026-08-01 12:48

   llm-d 跨三大 GPU 厂商实现异构推理服务 # llmd # ai https:// twp.ai/E5EnOT

   Heterogeneous inference serving across three GPU vendors with llm-d # llmd # ai https:// twp.ai/E5EnOT

   链接 llm-d.ai/…/heterogeneous-inference-3-vend…