PulseAugur
中
实时 22:29:07
English(EN) What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s

Snapdragon迷你PC可在CPU上高效运行大型语言模型

一位用户成功在一台价格实惠的联想IdeaCentre迷你PC上运行了大型语言模型,该PC配备了Snapdragon X1-26-100处理器和32GB内存。尽管缺乏GPU加速,该系统仍取得了令人印象深刻的性能,使用基于CPU的推理和定制的ARM优化,以约30 token/秒的速度运行了一个35B参数的编码模型,并以17-18 token/秒的速度运行了一个80B模型。用户探索了将模型专家缓存到RAM和从NVMe流式传输等技术,以克服内存限制,甚至尝试连接两台这样的PC来运行更大的模型。 AI

影响 展示了消费级硬件在本地运行大型语言模型方面日益增长的能力,可能降低了AI实验的门槛。

排序理由 用户级别的在消费级硬件上运行LLM的探索,并非前沿发布或重要的行业事件。

在 r/LocalLLaMA 阅读 →

AI 生成摘要 · Google Gemini · 来自 1 个来源。 我们如何撰写摘要 →

Snapdragon迷你PC可在CPU上高效运行大型语言模型

本文如何被排名

Signal score
3 / 100
Composite score across the factors below. Higher = stronger signal that this story matters right now.
Newsworthiness bucket
Tool
用户级别的在消费级硬件上运行LLM的探索,并非前沿发布或重要的行业事件。
Source corroboration
Single-source cluster
Only one publisher covered this so far. Single-source stories can still rank when the publisher is high-authority, but they lack cross-source corroboration.
Topics
infra, product
Editorial topic classification. Feeds into how the story surfaces on /topic/<slug> hub pages and into the per-entity coverage mix.
AI-industry relevance
High
Clearly on-topic for AI-industry coverage.
Story freshness
Breaking (< 6h)
Fresh story with cross-source coverage still developing. Ranking may shift as more sources report.

完整方法见我们的编辑标准。

报道来源 [1]

  1. r/LocalLLaMA TIER_1 English(EN) · /u/WorldlyAd7946 ·

    What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 tok/s

    <table> <tr><td> <a href="https://www.reddit.com/r/LocalLLaMA/comments/1x6stod/what_a_300_35w_snapdragon_mini_pc_can_actually_do/"> <img alt="What a £300, 35W Snapdragon mini PC can actually do 👀: 35B coder at ~30 tok/s, and an 80B model that doesnt even fit in RAM running at 17 …