PulseAugur / Brief
EN
LIVE 12:57:40

Brief

last 24h
[2/2] 221 sources

Multi-source AI news clustered, deduplicated, and scored 0–100 across authority, cluster strength, headline signal, and time decay.

  1. TOOL · Modal blog English(EN) ·

    How we achieved truly serverless GPUs

    Modal has developed a system to achieve truly serverless GPUs for AI inference, addressing the challenge of rapidly scaling resources to meet variable demand. Their approach involves maintaining cloud buffers of idle GPUs, a custom filesystem for lazy container image serving, and efficient checkpoint/restore mechanisms for both CPU and GPU processes. This engineering effort, developed over five years, reduces AI inference replica scaling time from tens of minutes to mere seconds, aiming to maximize GPU Allocation Utilization. AI

    How we achieved truly serverless GPUs

    IMPACT Enables faster, more efficient scaling of AI inference workloads, potentially lowering costs and improving resource utilization.

  2. RESEARCH · arXiv cs.CL Deutsch(DE) · · [3 sources]

    FastKernels: Benchmarking GPU Kernel Generation in Production

    Researchers have introduced FastKernels, a new benchmark designed to better evaluate GPU kernel generation agents used in production LLM inference. Existing benchmarks are misaligned with real-world systems, leading agents to produce kernels that perform poorly outside of testing environments. FastKernels aims to bridge this gap by serving as a production-grade inference framework that mirrors real-world deployment needs and covers a vast majority of HuggingFace Transformers architectures. AI

    IMPACT Addresses a critical bottleneck in LLM inference by improving the alignment of GPU kernel generation benchmarks with production systems.