Metal
PulseAugur coverage of Metal — every cluster mentioning Metal across labs, papers, and developer communities, ranked by signal.
- used by Apple Silicon 90%
- used by Mlx 90%
- used by Off Grid AI Desktop 90%
- used by Llama-2-7B-Chat-GGUF 90%
- used by Llama-2-70B-Chat-GGUF 90%
- used by Llama-2-13B-Chat-GGUF 90%
- developed by Apple Silicon 70%
- used by Vulkan 70%
- used by macOS 70%
- competes with Mlx 70%
- competes with Apple Silicon 70%
- affiliated with Apple Silicon 50%
11 day(s) with sentiment data
Apple Silicon's Metal API gaining traction for local LLM inference
Multiple recent articles highlight the increasing use of Apple Silicon's Metal API for local LLM inference. Salvatore Sanfilippo's ds4.c engine and the LM Studio guide both point to Metal as a key enabler for running large models on Macs. This suggests a growing ecosystem and optimization efforts around Metal for AI workloads on Apple hardware.
Apple to announce enhanced Metal support for AI/ML in upcoming WWDC
Given the recent focus on Metal for local LLM inference on Apple Silicon, it's plausible Apple will announce significant enhancements or new features for AI/ML development using Metal at the upcoming WWDC. This could include improved performance, new APIs, or better integration with popular ML frameworks.
Cross-platform GPU virtualization for AI is an emerging trend
The project connecting an NVIDIA GPU to a MacBook Air via a Linux VM demonstrates a novel approach to leveraging hardware across different operating systems for AI tasks. This workaround, while currently slower than native solutions, indicates a potential future direction for utilizing specialized hardware in environments with limited native driver support.
-
Mastodon post features "metal girls" art and metal music hashtags
This cluster contains a single item from Mastodon, a social media platform. The post is a collection of hashtags related to "metal girls" and various subgenres of metal music, alongside tags for art, illustration, and d…
-
NobodyWho and Cactus: On-Device LLM Engines Compared
A technical comparison highlights two on-device LLM inference engines, NobodyWho and Cactus, detailing their differences in engine design, model format, hardware acceleration, and licensing. NobodyWho utilizes llama.cpp…
-
llama.cpp bug causes non-deterministic results for M-RoPE embedding batches
A bug in the llama.cpp library causes incorrect results when processing embedding batches for M-RoPE models like Qwen3.5 and Qwen2.5-VL. The issue stems from a heap buffer overflow where the library reads past the alloc…
-
New macOS app Radiant Canvas offers faster local AI image generation
A new native macOS application called Radiant Canvas has been developed for local image inference using models like Krea 2, FLUX.2, and Qwen on Apple Silicon. The app is built with C++20, Objective-C++, and Swift, utili…
-
Research reveals silently defective LLM artifacts in public registries
A new research paper highlights significant issues with the integrity of large language model (LLM) artifacts available in public registries. The study found that 1.6% of official artifacts from Ollama and some communit…
-
llama.cpp b10835 fixes CUDA FlashAttention divergence on NVIDIA GPUs
The llama.cpp project has released build b10835, which addresses a critical bug in its f16 FlashAttention implementation on CUDA backends. This update resolves divergence issues that could lead to instability or errors …
-
Hermes Desktop simplifies local AI model setup with one-click installation
Nous Research has launched Hermes Desktop, a free, open-source application that simplifies the process of setting up and running open-weight AI models locally. The software automatically detects a user's hardware, selec…
-
llama.cpp releases updates with performance and stability fixes · 9 sources tracked
The llama.cpp project has released several updates, including version 0.4.1, which addresses various performance and stability issues across different platforms. Notable changes include optimizations for SYCL backends, …
-
Apple to unveil foldable iPhone Ultra next week; Mario Kart 64 playable on iPhone
Apple is reportedly planning to unveil its foldable iPhone Ultra next week during a September event. While the announcement is anticipated, the exact shipping date for the new model remains uncertain. Separately, a meth…
-
Perplexity open-sources Lily inference engine for Apple Silicon
Perplexity has open-sourced Lily, a specialized inference engine built with Rust and Metal for running the Qwen3.6-35B-A3B model on Apple Silicon. This engine is designed for narrow hardware optimization, achieving up t…
-
New METAL framework enhances federated video domain adaptation
Researchers have introduced METAL, a novel framework designed to improve Federated Video Domain Adaptation (FVDA). This approach addresses the challenge of aligning temporal information across distributed, non-IID video…
-
Developer ships Three.js game to iOS App Store after fixing rendering and GPU issues
A developer successfully shipped a Three.js game to the iOS App Store, running within a WKWebView via Capacitor without network calls. The process involved overcoming several silent failures, including an incorrect defa…
-
On-device AI gains traction with new SDKs and CCTV integration
The NobodyWho library is enabling developers to integrate large language models (LLMs) directly into applications for on-device AI, offering benefits like offline functionality, enhanced privacy, and reduced latency. Th…
-
Apple refreshes Mac desktops for local AI development
Apple has refreshed its Mac mini and Mac Studio desktop computers, emphasizing their capabilities for local AI development and inference. The updates introduce new chips, including the M6 and M5 Ultra, designed to enhan…
-
WebGPU unlocks browser GPU power for massive parallel computing
WebGPU is a new web API that allows developers to leverage the massive parallel processing power of graphics processing units (GPUs) directly from the browser. Unlike its predecessor, WebGL, which was limited to renderi…
-
Developer builds local LLM server with auto VRAM model selection
A developer has created a local LLM server using FastAPI and llama.cpp that automatically selects the appropriate GGUF model based on available GPU VRAM. This setup allows users to run various models, from 7B to 70B par…
-
Local LLM Server Mimics OpenAI API, Auto-Selects Models by VRAM
A developer has created a local LLM server that provides an OpenAI-compatible API, allowing users to run various GGUF models on their own hardware. The system utilizes llama.cpp for inference and FastAPI for the server,…
-
Local LLM server uses VRAM routing for efficient model selection
A technical guide demonstrates how to set up a local large language model server using llama.cpp and FastAPI. The system features VRAM-aware routing, allowing it to automatically select the most suitable LLM based on av…
-
4DAnyone model generates 4D video from single input
A new research paper introduces 4DAnyone, a model capable of generating multi-view videos from a single casual monocular video. This technology enables subsequent 4D Gaussian Splatting (4DGS) reconstruction, allowing fo…
-
User runs 465GB DeepSeek V4-Pro LLM on Mac Studio
A user details how they successfully run a 465GB LLM, DeepSeek V4-Pro, on a Mac Studio M3 Ultra with 512GB of unified memory. The setup prioritizes cost-effectiveness over raw speed, utilizing Apple Silicon's unified me…