The author is setting up a new personal blog on the Mastodon platform, utilizing the ActivityPub protocol. This move is part of a personal challenge to code using local, smaller Large Language Models (LLMs) rather than relying on cloud-based giants. The author details their hardware limitations, specifically a GTX 1060 GPU, and their choice of the Qwen 3.6 35B A3B model, particularly the Tiel Coder variant, for its balance of performance and resource requirements. They achieved a 96K token context window with CPU offloading and a decoding speed of 15 tokens per second. AI
IMPACT Demonstrates feasibility of running capable LLMs on consumer hardware, potentially lowering barriers for AI experimentation.
RANK_REASON The item is a personal blog post detailing a technical challenge and setup, rather than a primary release or significant industry event.
Read on Mastodon — fosstodon.org →
- ActivityPub
- Emacs Lisp
- GTX 1060
- lor.sh
- Mastodon
- mastodon.ml
- nginx
- Org-mode
- Qwen 3.6 35B A3B
- RTX 5090
- trump
- Truth Social
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →