DeepSeek 开发了一个自定义内核栈 DeepGEMM 和 TileLang，其性能不仅媲美，而且超越了 NVIDIA 的 cuBLAS。此自定义实现实现了比特级确定性和批次不变性，解决了其他工作负载平衡策略（如 splitK 或 split-KV）中常见的非确定性输出问题。这项创新在于其浮点数学方法，确保了调试和训练结果的一致性。 AI
影响 DeepSeek 的自定义内核栈提供了优于标准库的潜在性能优势，这可能会影响未来的 AI 基础设施开发和优化策略。
排序理由 该集群详细介绍了 AI 模型训练中自定义内核开发的技术创新，包括性能基准和技术解释，这符合研究级别的披露。
AI 生成摘要 · Google Gemini · 来自 4 个来源。 我们如何撰写摘要 →
完整方法见我们的编辑标准。
So why doesn't replacing cuBLAS cost them performance? Because DeepSeek's custom kernel stack (DeepGEMM + TileLang) is actually faster. Pre-compiled cuBLAS leaves real wins on the table: build-your-own with many more tile sizes available for better SM occupancy, JIT compilation
DeepSeek V4's answer: end-to-end bitwise deterministic and batch invariant kernels. They replaced all uses of cuBLAS with their own custom implementations in DeepGEMM. Instead of using atomics to accumulate partial results in split-reduction workloads, they write partials to
Common workload-balancing scheduling strategies such as splitK or split-KV give nondeterministic output bits for the same inputs. This makes debugging training failures nearly impossible -- you can't reproduce a loss spike. For inference, some kernels can give different results
Floating point math is not associative! And many of the highest performance kernels split the workload among SMs and accumulate partial results in a nondeterministic order. Many AI labs just accept this, or pay a huge performance penalty for determinism. DeepSeek decided to do ht…