Developer creates C#-native Ollama replacement for LLM inference

A developer has created a new inference server for Large Language Models (LLMs) entirely in C# using SpawnDev.ILGPU.ML. This server is designed to be a drop-in replacement for Ollama, supporting Ollama's API and reading models directly from the Ollama cache without re-downloading. While still in early development, its performance for interactive chat is comparable to Ollama, with token generation speeds approaching those of the established llama.cpp backend. The project aims to provide a fully C#-native solution for running LLMs, including tokenizer, dequantization, and attention mechanisms, with GPU kernels transpiled from C#. AI

IMPACT Offers a C#-native alternative for local LLM inference, potentially simplifying integration for .NET developers.

RANK_REASON This is a third-party tool that integrates with existing LLM infrastructure, rather than a direct release from a frontier lab.

Read on dev.to — LLM tag →

• C#
• Claude CLI
• Codex
• Continue
• GGUF
• ILGPU
• llama.cpp
• Ollama
• OpenAI
• qwen2.5-coder:7b
• Pi
• SpawnDev.ILGPU.ML

• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →