Language model inference runs inside Linux kernel via eBPF

A novel experiment has demonstrated the feasibility of running a language model's core computations within Linux's eBPF (extended Berkeley Packet Filter) programs. The project, named qwen3-ebpf, successfully executed the 28 decoder layers of the Qwen3-0.6B model, including numerical operations like RMSNorm, attention, and MLP, entirely within the kernel. While this in-kernel inference is not a replacement for traditional CPU or GPU processing due to its slower speed (approximately 1.2 seconds per token), it showcases the potential for eBPF to handle complex numerical tasks beyond its typical observation and filtering roles. The implementation carefully manages state through BPF maps and uses fixed-point integer arithmetic to satisfy the Linux verifier's constraints. AI

IMPACT Demonstrates novel kernel-level inference capabilities, potentially enabling new security or performance monitoring tools.

RANK_REASON The item describes a technical experiment and implementation of a language model within a specific technical framework (Linux eBPF), which constitutes research. [lever_c_demoted from research: ic=1 ai=1.0]

Read on dev.to — LLM tag →

• bfloat16
• bpf()
• bpf_prog_test_run_opts
• central processing unit
• eBPF
• graphics processing unit
• Linux
• Qwen3-0.6B
• qwen3_attention
• qwen3_batch
• qwen3-ebpf
• qwen3_norm
• qwen3_rope
• qwen3_silu
• qwen3_vector
• RMSNorm

• infra
• model release

AI-generated summary · Google Gemini · from 1 sources. How we write summaries →