Pulse

The redb.Route integration framework has released version 3.1.0, introducing two new transports: redb.Route.Llm and redb.Route.Exec. The LLM transport allows developers to treat language models as addressable endpoints, similar to Kafka or HTTP, enabling seamless integration of LLM calls within existing integration workflows. This release also introduces the capability to define agent tools as routes with an `.AsLlmTool()` aspect, unifying AI functionalities within the framework's existing DSL and infrastructure. AI

Google Fi has announced significant upgrades to its international roaming services, including faster 5G speeds and reduced prices for data usage abroad. These enhancements aim to make international travel more convenient and affordable for its users. The changes are part of Google Fi's ongoing efforts to improve its global connectivity offerings. AI

Xbox is reportedly struggling to meet the demand for its new consoles, with speculation pointing to the ongoing AI boom as a contributing factor to hardware shortages. The article suggests that Microsoft's own involvement in the AI sector may be exacerbating the issue by driving up prices and demand for essential components like RAM and GPUs. This situation is prompting a reevaluation of Microsoft's "Project Helix." AI

A developer has created a Rust-native, CPU-only implementation of the LFM2.5-8B-A1B language model. This project, still in progress, has been published as a cargo crate and includes features like tool use callbacks. The implementation offers a decode speed of approximately 37 tokens/s on a Ryzen 7950x and can run on systems with as little as 16GB of RAM, with memory usage around 7GB. AI

A user has successfully configured a Jetson Orin NX for running the Hermes Agent, achieving impressive performance metrics. The build prioritizes silence and aesthetic appeal while delivering over 10 tokens/sec for text generation and 300 tokens/sec for prompt processing. The setup supports a context window of at least 65,000 tokens, with specific testing showing a Gemma 4 26B model achieving 10.21 tokens/sec at 60,000 tokens of context. AI

Excoms AI has launched its AI WiFi Survey Agent, now available on the Microsoft Commercial Marketplace. This tool allows users to upload floor plans, which the AI then analyzes to determine wall materials and predict wireless coverage across various frequencies. The agent generates a branded PDF report with recommended access point placements, eliminating the need for on-site visits or specialized equipment. AI

A user discovered that Google's Gemini AI Pro has a hidden rate limit that is easily triggered by frequent API calls, even for simple tasks. This limit is not clearly documented and can be hit within hours of consistent usage, unlike other more predictable usage caps. The user found that making many small, rapid API requests, rather than complex or long-running ones, was the primary cause of hitting these limits. AI

Smista.ai has released its second development update, detailing the creation of smista-storage. This component is designed to manage AI user sessions, which are complex structures involving messages, tool calls, and routing decisions. The team selected SurrealDB for its flexibility, enabling both local, embedded use and future scalability for a SaaS offering, adhering to their 'local-first, but not local-only' principle. AI

OpenRouter is a platform designed to simplify access to various AI models. It allows users to compare the costs and performance of different models and integrate them via API. The service aims to help users select the most cost-effective AI options for their needs. AI

A new research paper proposes an alternative to the Transformer architecture, which powers most large language models. This alternative aims to address the significant computational cost associated with Transformer inference. If successful, this could potentially reduce the massive financial investment currently driving the AI industry. AI

Apple is developing its "Domesticated AI" services, which will leverage on-device models and a secure Private Cloud Compute infrastructure. These AI capabilities will be integrated with Apple's cloud services, potentially offered for free to most users or as part of an Apple Cloud upgrade for expanded token usage. The system aims to ground personal context and can be extended to include family members' contexts. AI

A user found LocalAI to be a superior alternative to Ollama and AnythingLLM for running AI models locally. They highlighted LocalAI's all-in-one solution for model downloads, backends, and a WebUI, all manageable within Docker with GPU acceleration. The user also noted impressive performance, achieving 95 tokens/second on their RX7900XTX GPU. AI

Cloudflare is leveraging its own infrastructure to defend against advanced AI-powered cyber threats. The company is using its extensive network and security architecture as a testing ground, or "customer zero," to develop and deploy defenses against sophisticated attacks. This proactive approach aims to stay ahead of evolving cyber threats that utilize frontier AI models. AI

Gigabyte is presenting its comprehensive AI infrastructure solutions at COMPUTEX 2026. Their display spans from large-scale rack systems to practical deployment applications. The company aims to highlight its end-to-end capabilities in the AI hardware sector. AI

Cloudflare has integrated threat intelligence into its Web Application Firewall (WAF), allowing users to block attacks based on the attacker's name and their targeted industries. This move shifts security defenses from a passive approach to a more contextual one, aiming to provide greater visibility into threats. The new feature is expected to change how security operations manage information asymmetry in the face of evolving cyber threats. AI

A pull request for the llama.cpp project introduces optimizations for k-quantized models, significantly improving prefill speeds. The changes focus on the matrix multiplication (matmul) operations for various quantization levels, including Q4, Q5, and Q8. Benchmarks on an M2 Pro chip show speedups of up to 3.78x for certain quantizations, enhancing the performance of local large language models. AI

SpaceX has unveiled its initial design for an AI-powered satellite. This satellite is intended to enhance SpaceX's Starlink internet constellation by integrating artificial intelligence capabilities directly into its space-based infrastructure. The move signifies a significant step in merging AI technology with satellite operations for improved performance and functionality. AI

Apple has released MLX LM Server, a new tool designed to enhance the performance of large language models on Mac hardware. It leverages the M5 chip's neural accelerators for faster prompt processing and employs continuous batching to manage multiple requests concurrently. For extremely large models, the server supports distributed inference across multiple Macs using Thunderbolt RDMA. AI

A user discovered that the default pipeline parallelism in llama.cpp may be wasting VRAM without providing any speed benefits. By compiling llama.cpp with the flag -DGGML_SCHED_MAX_COPIES=1, users can avoid this unnecessary VRAM allocation. This optimization is particularly relevant when all model layers are offloaded to the GPU. AI

Microsoft has unveiled the Surface RTX Spark Dev Box, a compact PC specifically designed for AI development. The company also announced Scout, an autonomous agent built on the OpenClaw framework with MCP support. These announcements highlight Microsoft's continued investment in AI infrastructure and agentic AI capabilities. AI

A Reddit user has identified issues with Google's quantization process for large language models, specifically noting that the llama-quantize function is hardcoded incorrectly and misaligns block groups. The user suggests that the unsloth Q4_K_XL quantization method is a more reliable alternative for now. A patch is reportedly in development to address these quantization errors. AI

Apple has released its Core AI Framework, a new set of tools for developers to integrate machine learning capabilities into their applications. The framework is detailed in documentation available on Apple's developer portal. This release aims to empower developers to build more intelligent and responsive apps across Apple's ecosystem. AI

Apple is enhancing its developer tools to better integrate AI capabilities across various platforms. Developers can now route AI tasks between on-device models, Apple's Private Cloud Compute, and external servers. This integration aims to deepen AI functionality within Siri and offer developers more flexibility in processing, though device memory and regional availability remain limitations. AI

Users on the r/LocalLLaMA subreddit are discussing the most cost-effective hardware configurations for running the GLM-5.1 and Kimi K2.6 large language models. Participants are seeking advice on achieving inference speeds of 15-20 tokens per second with minimal expense. Suggestions range from high-end consumer GPUs like the RTX 5090 paired with substantial RAM, to professional-grade hardware such as Threadripper CPUs, Mac Studio Ultra machines, or multiple V100 GPUs. AI

Lookspan has updated its billing system to specifically track and charge for reasoning tokens. This change ensures that if a model's pricing includes a distinct rate for reasoning, those specific tokens will be billed accordingly, preventing double-charging with general output tokens. The update aims to provide more precise cost-per-span calculations for models that utilize reasoning capabilities. AI

Apple has released its Core AI framework, a new set of tools designed to help developers integrate artificial intelligence capabilities into their applications. The framework provides access to on-device machine learning models and functionalities, enabling richer and more responsive AI experiences within the Apple ecosystem. Developers can leverage Core AI to build features such as image analysis, natural language processing, and predictive text directly into their iOS, macOS, and other Apple platform applications. AI

Uber has already exhausted its artificial intelligence budget for 2026 by April, indicating a significant overspend on AI services. Employees are reportedly monitoring token usage closely, reminiscent of early internet users rationing data. This situation highlights the unsustainable per-token pricing model for AI inference, suggesting that current costs will not persist as the technology evolves. AI

A healthcare organization experienced an 8-10% monthly increase in AI token usage, resulting in $6 million in unexpected expenses. This significant cost overrun has prompted the organization to adopt AI gateways and observability tools for better spend attribution. The situation highlights a broader industry challenge in tracking AI expenditures, with a call for standards in tokenomics to improve cost transparency. AI

A user successfully set up and self-hosted AI models on their home server, finding the process easier than anticipated. The primary advantage of this setup is enhanced privacy. However, the main disadvantage is the slow performance, attributed to hardware limitations. AI

A user shared optimization tips for running the DeepSeek v4 Flash model locally, achieving nearly 200 tokens per second on a Hopper system. By utilizing specific quants from Canada-Quant and patching the MTP code in vLLM, the user managed to significantly improve inference speed. The post also details the cost implications, noting that electricity costs for token generation currently exceed revenue. AI

Canonical has introduced a new approach to AI agent development using Ubuntu, leveraging LXD "containervisors" and snap packaging. This system creates isolated sandboxes for LLM agents, granting them controlled access to resources like GPUs and specific files while preventing access to sensitive personal data. The initiative aims to simplify the installation and execution of AI agents while enhancing security through resource limitation. AI

Meta is reportedly planning to establish data centers within temporary structures, a move detailed by GIGAZINE. Concurrently, ZDNET Japan reports that Vertiv is integrating NVIDIA's digital twin technology into its AI data center designs. These developments highlight evolving strategies in AI infrastructure, focusing on both novel deployment methods and advanced design tools. AI

Amazon SageMaker is now supporting end-to-end encrypted machine learning inference using fully homomorphic encryption (FHE). This advancement allows for secure processing of sensitive data without decryption, enhancing privacy in AI applications. The integration builds upon previous discussions about FHE's potential for secure ML inference. AI

Luce Spark is a new open-source system that allows large Mixture-of-Experts (MoE) language models, specifically 33-35 billion parameters, to run on a single 16GB GPU. It achieves this by intelligently keeping only the currently active experts on the GPU, while the rest are stored in system RAM and swapped in as needed. This method avoids the performance penalty typically associated with offloading, enabling models that would otherwise not fit to run efficiently. AI