NInfer
PulseAugur coverage of NInfer — every cluster mentioning NInfer across labs, papers, and developer communities, ranked by signal.
- 2026-10-02 product_launch A new AI inference backend named NInfer has been released, noted for its high performance. source
- 2026-08-29 product_launch A fork of the NInfer inference engine was released, enabling a 1 million token context window for the Qwen-3.8 27B model. source
- 2026-08-14 product_launch NInfer has released day-zero support for the Qwen3.8-27B model, including significant engine improvements. source
1 day(s) with sentiment data
-
NInfer AI backend offers significant speed improvements over llama-server
A new AI inference backend called NInfer has been released, offering significantly faster performance compared to existing solutions like llama-server. Users report speeds of 130-180 tokens per second, with concurrent c…
-
qwen3.6 35ba3b model hits 600 tokens/sec on RTX Pro 6000
A user on Reddit's r/LocalLLaMA subreddit reported achieving 600 tokens per second for single requests using the qwen3.6 35ba3b model. This performance was achieved with the NInfer framework running on an Nvidia RTX Pro…
-
New Ninfer Studio harness released for Ninfer inference engine
A developer has created Ninfer Studio, a new harness designed to work specifically with the Ninfer inference engine. This tool provides both a coding harness and a chat interface, allowing users to easily customize and …
-
NInfer fork boosts LLM context to 555k with 4-bit KV cache
A fork of the NInfer project has been developed, introducing significant improvements to context length and memory management for large language models. This fork features a custom 4-bit KV cache that reduces VRAM usage…
-
Local LLM generator learns to self-correct code based on video output
A developer has enhanced their "auto_demo_scener" project, which uses local LLMs to generate demoscene effects. The latest update integrates Ninfer support and a new feature allowing the LLM to review its own generated …
-
NInfer, llama.cpp, vLLM speed and quality compared for Qwen3.8-27B
A user conducted a performance comparison of three inference engines—NInfer, llama.cpp, and vLLM—on a single RTX 5090 GPU using the Qwen3.8-27B model. The evaluation focused on quality and speed for a production content…
-
NInfer fork enables 1M context for Qwen-3.8 27B on dual 5090s
A fork of the NInfer C++20/CUDA inference engine has been developed to support a 1 million token context window for the Qwen-3.8 27B model. This enhanced version, running on dual 5090 GPUs, achieves significantly higher…
-
NInfer boosts local LLM performance to 220 tokens/sec on 5090 GPU
A user on Reddit's r/LocalLLaMA subreddit shared their positive experience using NInfer with a 5090 GPU to run a 27 billion parameter model. They reported achieving significantly higher throughput, with speeds averaging…
-
NInfer fork enables 2x performance boost for Qwen3.6-35B on CMP170HX hardware
A user has successfully forked the NInfer project to enable it to run on CMP170HX hardware, achieving a twofold performance increase for the Qwen3.6-35B model. This modification involved adjusting CUDA kernels and compi…
-
Qwen model made more concise with "Sharp" prompt overlay for NInfer
A user has developed a method to make the Qwen model more concise by applying a "Sharp" system prompt overlay to the NInfer inference engine. This modification reduces output tokens by 42% without impacting speed or cor…
-
Qwen3.8-27B model hits 880 tok/s on single RTX 5090 with NInfer engine
A user has achieved impressive performance with the Qwen3.8-27B model on a single RTX 5090 GPU, reaching 880 tokens/second with 4-bit NVFP4 quantization and a full 262k context. This speed was attained using the NInfer …
-
NInfer update enables 350K token context for Qwen 3.8-27B on RTX 4090
A user has updated their fork of NInfer, a tool for running large language models locally, to support the Qwen 3.8-27B model. This update allows for context windows of up to 250-350K tokens on a single RTX 4090 GPU, wit…
-
NInfer adds day-zero support for Qwen3.8-27B model with engine improvements
The NInfer inference engine has released day-zero support for the new Qwen3.8-27B model, enabling local deployment and experimentation. This update brings significant engine improvements, including support for up to 8 c…
-
NInfer engine achieves 542 tok/s for Qwen3.6 on single RTX 5090
A new inference engine called NInfer, built from scratch in C++/CUDA, has been open-sourced, demonstrating impressive performance with the Qwen3.6-35B-A3B model. The engine achieved a sustained speed of 542 tokens per s…