Victor Gallego

New Runtimes and Benchmarks Boost LLM Inference on Apple Silicon

Researchers have developed new methods for optimizing large language model (LLM) inference on Apple Silicon. The first approach, BaseRT, is a native Metal runtime that achieves higher inference throughput than existing …