SK Hynix and SanDisk have collaborated to develop High Bandwidth Flash (HBF), a new memory tier designed to address the memory wall challenges in AI inference. HBF places large-capacity NAND flash memory close to various xPUs, including GPUs, aiming to alleviate bottlenecks caused by data movement. This specification, developed with input from Google and Tenstorrent, positions HBF between HBM and SSDs, managing large datasets like model weights, KV Cache, and agent workloads through a Base Die and host software. While HBF offers high aggregate read bandwidth through parallel access across multiple channels, it retains NAND's inherent limitations in write endurance and management, shifting complexity to the software stack. AI
IMPACT HBF aims to improve AI inference performance by providing a new memory tier closer to processors, potentially enabling larger models and faster processing.
RANK_REASON This item details a new specification for a memory technology (HBF) developed by industry players, positioning it within the AI hardware ecosystem. [lever_c_demoted from research: ic=1 ai=1.0]
AI-generated summary · Google Gemini · from 1 sources. How we write summaries →
