Nifer inference engine achieves 720t/s on Qwen 3.6 35B model

A new inference engine called Nifer has been released, reportedly achieving speeds of 550-720 tokens per second on a Qwen 3.6 35B model. This performance, described as "insane" by users, is achieved without complex batching or parallel agents, and supports a context window of 250k tokens. The engine is specifically optimized for the RTX 5090 GPU and is available on GitHub, though initially for Linux with potential for Windows builds. AI

IMPACT Potentially enables significantly faster local inference for large language models.

RANK_REASON Release of a new inference engine optimized for specific hardware.

Read on r/LocalLLaMA →

• Cerebras
• Qwen3.6 27b
• Qwen 3.6 35B
• RTX5090

• infra
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →