TabbyAPI
PulseAugur coverage of TabbyAPI — every cluster mentioning TabbyAPI across labs, papers, and developer communities, ranked by signal.
2 day(s) with sentiment data
-
Ten local LLM inference engines evaluated for 100GB RAM, dual-GPU rigs
A blog post evaluates ten local inference engines for their suitability on a machine with approximately 100 GB of RAM and two 16 GB GPUs. The author details the lineage of these engines, noting that several are derived …
-
ExLlamaV3 and TabbyAPI enable local LLM deployment on consumer hardware
ExLlamaV3 and TabbyAPI have been released, offering new capabilities for running large language models locally on consumer hardware. ExLlamaV3 introduces an optimized quantization method called EXL3, based on QTIP, whic…
-
Four large local LLMs benchmarked on RTX 5090 with 128GB RAM
A technical benchmark compares four large local language models (LLMs) running on a single RTX 5090 GPU with 128 GB of RAM. The models tested include DeepSeek-V4.1-Flash (748B parameters), GLM-5.3-Flash (321B parameters…
-
GLM-5.3-Flash (321B) runs locally on single RTX 5090 via Project Maya
A technical guide details how to run the large GLM-5.3-Flash (321B) language model locally on a single RTX 5090 GPU with 128 GB of RAM. The process involves using Project Maya, a fork of the Strata runtime, which intell…
-
Qwen3.8 Flash Next model optimized for local inference on diverse hardware
The Qwen3.8 Flash Next model is being optimized for local inference across various hardware configurations. One user achieved 400 tokens/sec on an RTX 6000 with a fork of NInfer, while another demonstrated running the m…
-
GLM-5.3-UNCENSORED EXL3 3.0bpw model released with ExLlamaV3 quantization
A quantized version of the GLM-5.3 model, named GLM-5.3-UNCENSORED EXL3 3.0bpw, has been released on Hugging Face. This model is based on the dealignai/GLM-5.3-UNCENSORED-FP8, which itself is a weight-edited variant of …
-
ExLlamaV3 praised for superior performance in local LLM deployment
A Reddit user is advocating for ExLlamaV3, a software that they believe is underrated for running large language models locally. They highlight its superior quantization quality, lower KLD metrics, and faster performanc…
-
EXL3 model loses traction in r/LocalLLaMA community despite performance
The EXL3 model, an alternative to llama.cpp for running large language models, appears to be losing traction within the r/LocalLLaMA community. While EXL3 offers strong performance in tokens-per-second and unique compre…