mlx-lm
PulseAugur coverage of mlx-lm — every cluster mentioning mlx-lm across labs, papers, and developer communities, ranked by signal.
4 day(s) with sentiment data
-
OptiQ overcomes MTP integration hurdles for Apple Silicon speedup
The development team at OptiQ encountered significant challenges integrating Multi-Token Prediction (MTP) into their OptiqEngine for Apple Silicon, despite initial expectations of a quick implementation. They identified…
-
New speculative decoding method optimizes Mixture-of-Experts models
Researchers have developed a new method for speculative decoding in Mixture-of-Experts (MoE) models, focusing on optimizing expert byte requests and kernel bandwidth. Their approach aims to reduce the computational cost…
-
Nanbeige4.3-33B model performance compared across three inference engines on MacBook Pro M5 Max
A user on Reddit compared the performance of the Nanbeige4.3-33B model across three different inference engines on a MacBook Pro M5 Max. The comparison focused on prefill and decode speeds, time to first token, latency,…
-
Meta releases Muse Spark Open, a large open-weight MoE model
Meta has released Muse Spark Open, an 816 billion parameter Mixture-of-Experts model with open weights, achieving the highest score among US open-weight models on Artificial Analysis's Intelligence Index. While it excel…
-
Meta releases Muse Spark Open, a high-scoring open-weights model
Meta has released Muse Spark Open, an 816B-total, 42B-active Mixture-of-Experts model with open weights, achieving a high score on Artificial Analysis's Intelligence Index. While it excels in some areas like code genera…
-
Local LLM Deployment: Memory, Speed, and Runtime Considerations
When running large language models locally, users often face issues with hardware capacity and performance metrics. It's crucial to differentiate between memory fit, which can be calculated, and speed, which requires sp…
-
LLMeter CLI measures LLM performance on local hardware
LLMeter is a new command-line interface tool designed to measure the performance of large language models (LLMs) on a user's specific hardware and configuration. Unlike traditional leaderboards that test models on optim…
-
jialinyyzz/humanizer model integrates with Transformers, MLX, and local apps
The jialinyyzz/humanizer model is now available for use with various popular AI tools and libraries. Instructions are provided for integrating it with Hugging Face's Transformers, MLX, and llama.cpp, enabling users to r…
-
Edge0 releases 35B MoE LLM for low-memory devices
Edge0 has released a preview of its Edge0-35B-A3B model, a 35 billion parameter sparse Mixture-of-Experts (MoE) large language model designed to run efficiently on devices with limited memory. The model requires under 3…
-
Perplexity open-sources Lily inference engine for Apple Silicon
Perplexity has open-sourced Lily, a specialized inference engine built with Rust and Metal for running the Qwen3.6-35B-A3B model on Apple Silicon. This engine is designed for narrow hardware optimization, achieving up t…
-
Perplexity open-sources Lily inference engine for Apple Silicon
Perplexity has open-sourced Lily, a local inference engine designed for hybrid compute within its Perplexity Computer product. Lily is specifically optimized for running Qwen3.6-35B-A3B models on Apple silicon, treating…
-
Apple Silicon LLM inference hampered by fragmented software stack
Running large language models efficiently on Apple Silicon is hampered by a fragmented and immature software ecosystem. Unlike the integrated optimizations available on NVIDIA's CUDA platform, Apple's platform lacks a u…
-
Gemma models on Apple Silicon suffer silent cache bug, slowing local AI
A cache bug affecting Gemma models on Apple Silicon has been identified, causing significant slowdowns in local AI agent performance. The issue stems from Gemma's sliding-window attention mechanism, which, when exceedin…
-
FusionML boosts Apple Silicon AI inference with CPU-GPU co-execution
Researchers have developed FusionML, a system that optimizes transformer inference on Apple Silicon by co-executing CPU and GPU operations. By addressing limitations in MLX's scheduler that caused serialization, FusionM…
-
Gemma 4 12B local AI model requires configuration tweaks for optimal performance
Google's Gemma 4 12B model shows promise for local AI setups, but users report that default configurations in tools like LM Studio can hinder its reasoning capabilities. Specific adjustments to Jinja templates and sampl…