Developer builds LLM-free terminal assistant after failed custom model experiments

A developer explored creating a terminal assistant without relying on large language models (LLMs) due to concerns about rising AI costs. Initial attempts involved training custom transformer and Mamba models from scratch on personal hardware, but these proved inefficient and produced unsatisfactory results. The developer then experimented with existing open-weight models via Ollama, finding them too slow for practical use, especially with limited VRAM. Ultimately, the project pivoted to a new approach, explicitly avoiding embeddings, machine learning, and LLMs. AI

IMPACT Demonstrates potential for efficient, non-LLM-based AI tooling for specific tasks, reducing reliance on costly large models.

RANK_REASON The item describes the development of a new tool (terminal assistant) and the developer's process, rather than a release from a major AI lab or significant industry event.

Read on Hacker News — AI stories ≥50 points →

• ETH Zurich
• GTX 1050 Ti
• Hugging Face
• i7-4790K
• Mamba
• nanoGPT
• Nvidia
• Ollama
• Pius Sieber
• Project Gutenberg

• product
• infra

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →