The llama.cpp project has released version b11813, which includes updates to its pre-tokenizer for the Pixtral 12B model, moving away from the Llama 3 tokenizer. This release also details compatibility and build options across various operating systems including macOS, Linux, Android, and Windows, with specific configurations for openEuler. Additionally, the Hugging Face model prism-ml/Ternary-Bonsai-2-9B-gguf is highlighted, with extensive instructions provided for its integration with llama.cpp and other popular inference tools like LM Studio, Jan, vLLM, and Ollama. AI
IMPACT Facilitates broader adoption of new models by improving compatibility and integration with popular inference tools.
RANK_REASON This cluster details updates to a software tool (llama.cpp) and the integration of a specific model (prism-ml/Ternary-Bonsai-2-9B-gguf) with various inference platforms.
Read on llama.cpp — Releases →
- Hugging Face
- Jan
- Linux
- Llama 3
- llama.cpp
- LM Studio
- macOS
- Microsoft Windows
- Ollama
- Pixtral 12B
- prism-ml/Ternary-Bonsai-2-9B-gguf
- Tekken
- vLLM
AI-generated summary · Google Gemini · from 2 sources. How we write summaries →