NVM Express
PulseAugur coverage of NVM Express — every cluster mentioning NVM Express across labs, papers, and developer communities, ranked by signal.
10 day(s) with sentiment data
-
Colibri engine enables 744B parameter LLMs on desktop via novel weight streaming
A new inference engine called Colibri allows users to run extremely large Mixture-of-Experts (MoE) models, such as those with 744 billion parameters, on standard desktop hardware. Instead of compressing the model to fit…
-
150B MoE model runs on laptop via NVMe storage, bypassing GPU
Researchers have demonstrated a 150-billion-parameter mixture-of-experts (MoE) model that can be streamed directly from NVMe storage on a standard developer laptop. This setup bypasses the need for a usable GPU, with th…
-
Colibri inference engine enables 744B models on consumer hardware
Colibri, a C inference engine, enables running large language models with up to 744 billion parameters on consumer hardware. It achieves this by storing dense layers in RAM and streaming routed experts from NVMe storage…
-
Ollama model reloading issue resolved with server-side keep-alive setting
A developer investigated why their local AI model was performing inconsistently, discovering that Ollama was reloading the model from disk frequently due to a default 5-minute idle timeout. This reloading process signif…
-
SSD vs. HDD: Speed vs. Storage Capacity Explained
Choosing between a 256GB solid-state drive (SSD) and a 1TB hard disk drive (HDD) depends on user needs, balancing speed against storage capacity. SSDs offer significantly faster performance for operating systems and app…
-
Amazon SageMaker HyperPod adds model caching to speed up inference
Amazon SageMaker HyperPod has introduced model caching to reduce inference cold starts. This feature pre-loads model weights and container images onto cluster nodes, allowing pods to access data from local NVMe storage …
-
Databricks boosts Lakebase Postgres speed with new compute cache
Databricks has enhanced the compute cache for Lakebase Postgres, aiming to improve performance and reduce latency for users. The company has implemented a local file cache (LFC) that works alongside Postgres's shared bu…
-
Debian 13 cloud installs vary widely in tools and vulnerability reporting
A comparison of Debian 13 installations across AWS, Azure, and Google Cloud Platform (GCE) revealed significant differences in default package availability and hardware toolsets. While a local Debian 13 install had 1,92…
-
KVMem virtualizes million-token AI agent workspaces on consumer GPUs
Researchers have developed KVMem, a system designed to manage large context windows for AI agents, enabling them to operate with up to one million tokens on consumer-grade GPUs. This virtualization technique stores over…
-
Kaleidescape launches 246TB SSD media server for luxury home theaters
Kaleidescape has launched its new Compact Terra Prime 246TB SSD media server, designed for high-end home theater systems. This server can store up to 2,300 high-bitrate 4K movies and supports 25 simultaneous 4K streams.…
-
VMware enhances memory tiering to combat high hardware costs
VMware is enhancing its memory tiering technology to address persistently high hardware and memory prices. This technology moves less frequently accessed data from expensive DRAM to faster NVMe storage, aiming to improv…
-
AI training bottlenecked by metadata, not bandwidth, demanding new storage solutions
AI training performance is often bottlenecked by storage, not raw bandwidth, due to the immense number of small metadata files generated during checkpointing and dataset operations. These metadata operations can overwhe…
-
SanDisk launches high-endurance NAS SSDs with 14 PBW rating
SanDisk has introduced new NAS-focused SSDs, the NAS 600 SATA and NAS 800 NVMe series, designed for continuous 24/7 operation. The top-tier NAS 800 7.68TB model boasts an impressive endurance rating of 14 PBW, allowing …
-
LLM storage latency tolerance shifts stepwise with GPU utilization
Storage latency tolerance for LLM inference does not decrease linearly with GPU utilization, but rather in a stepwise manner. At higher GPU utilization levels (around 90%), the compute queue saturates, making storage la…
-
Minisforum N5 Pro NAS Reviewed with Ryzen AI 9 Power
The Minisforum N5 Pro, a five-bay Network Attached Storage (NAS) device, has been reviewed. This NAS is powered by the Ryzen AI 9 HX PRO 370 processor and runs on Ubuntu 26.04 LTS. It features NVMe storage and 10-gigabi…
-
AirLLM enables 70B model inference on 4GB GPU by streaming layers from disk
AirLLM is a new project that enables running large language models, such as a 70B parameter model, on hardware with very limited VRAM, like a 4GB GPU. It achieves this by loading model layers sequentially from disk to t…
-
SSD speed impact on gaming performance analyzed across 11 titles
A recent analysis of 11 gaming titles reveals that while faster SSDs like PCIe 5.0 offer some improvements, the real-world impact on gaming performance is often marginal. The study compared SATA and NVMe SSDs, noting th…
-
KV Cache Prefetching Slashes LLM Inference Latency
A new prefetching strategy for KV Cache data has been developed, significantly reducing storage latency during large model inference. This method, tested on the Mingxin FX100 with a 480B model, improves inference throug…
-
NVMe adds virtualization for SSDs, boosting storage in virtualized environments
The NVMe specification has been updated to include virtualization capabilities for locally attached Solid State Drives. This enhancement aims to improve storage management and performance in virtualized environments. Th…
-
KV Cache tiering boosts LLM inference speed and cuts costs
A new approach to managing KV Cache in large language model inference suggests treating it as a high-frequency access subset within the warm storage tier, rather than in the traditional hot or cold tiers. This strategy,…