Quantr

New C-based inference engine Quantr explores running LLMs without GPUs

A new experimental LLM inference engine named Quantr has been developed in C to explore the intricacies of running large language models without relying on GPUs. The project focuses on quantized model weights, GGUF form…