I built a iOS app to benchmark GGUF models on your iPhone/iPad
A new free iOS application called GenBench has been released, allowing users to download, run, and benchmark GGUF models directly on their iPhones and iPads. The app utilizes llama.cpp and Metal for offline operation and measures performance metrics such as tokens per second, first-token latency, and peak memory usage. Users can also submit their scores to a global leaderboard to compare performance across different devices and models, including text and vision models. AI
IMPACT Enables users to easily test and compare the performance of various AI models directly on their mobile devices, fostering broader experimentation and understanding of on-device AI capabilities.