Krasis update: Qwen3.6-35B-A3B (Q4) at reading speed, 1x 8GB 3070 Mobile laptop (32GB RAM)
The Krasis LLM runtime has been updated to version 1.0, featuring a complete rewrite in Rust for improved performance and efficiency. This update removes Python from the critical execution path, leading to faster prefill and decode speeds. Krasis now supports Ampere (RTX 3000 series) GPUs and has optimized memory requirements, needing only 1x the quantized model size plus overhead in system RAM. AI
IMPACT Improved efficiency for running large LLMs locally, potentially lowering hardware barriers for advanced model usage.