llama.cpp项目为Mac硬件引入了新的Metal Performance Tensors (MTP)功能，显示出在token生成速度方面有潜在的提升。在M2 Ultra上的初步测试表明，虽然prompt处理速度保持一致，但在启用MTP的情况下，token生成速度可能会变得更加多变，尤其是在更长的上下文长度下。此外，该项目还解决了在隔离网络Mac上构建llama.cpp的问题，需要在构建过程中使用特定标志来禁用UI下载。 AI
影响 提高了在Mac硬件上本地LLM推理的性能和可用性。
排序理由 文章讨论了现有开源软件项目的改进和功能，而不是新的模型发布或重大的行业性事件。
AI 生成摘要 · Google Gemini · 来自 2 个来源。 我们如何撰写摘要 →
<h2> MTP </h2> <p>So I decided to test out the new MTP in llama.cpp on Metal using my M2 Ultra, and figured I'd toss the results up here. This isn't meant to show the maximum tps you can get on Mac hardware; I'd have run it on the M5 Max or M3 Ultra if that were the case. My goal…
<p>If you ever tried to run Llama.cpp on a MacOS device that doesn't have internet on it, you've probably hit the annoying GateKeeper errors that it's downloaded from the internet and you should delete it. Generally I just build from source to avoid that, but I ran into something…