This article compiles a list of 10 free resources for learning LLM inference engineering, covering essential topics like KV cache, batching, serving engines, and GPU kernels. The resources include a handbook from Modular, continuous batching explanations from Hugging Face, university lectures from Stanford and MIT, and deep dives into specific engines like vLLM and Mini-SGLang. It also points to GPU MODE lectures and Triton tutorials for further optimization knowledge. AI
IMPACT Provides a structured learning path for developers to optimize LLM inference performance.
RANK_REASON The article is a curated list of existing resources for learning a specific technical skill, rather than a new release or significant industry event.
- Aleksa Gordic
- CUDA
- Flashattention
- FlashInfer
- Google/Deepmind
- Hugging Face
- KV cache
- Mini-SGLang
- MIT
- Modular
- Nvidia
- SGLang
- Song Han
- Stanford University
- vLLM
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →