Brief

HPC-Ops has released a significant update to its open-source inference system, introducing five key operators. This upgrade addresses critical engineering bottlenecks such as attention latency, memory transfer costs, and cross-card communication on mainstream inference platforms. The new operators reportedly outperform existing open-source baselines in performance metrics, enhancing adaptability to dynamic workloads and supporting complex precision and performance fusion operators. AI

Researchers have developed SwiftCTS, a novel framework for optimizing clock tree synthesis in chip design. This system uses physics-informed surrogate models and gradient-boosted ensembles to achieve rapid predictions and Pareto optimization of power, wirelength, and timing skew metrics. SwiftCTS can adapt to new chip architectures with minimal calibration, significantly reducing prediction errors and enabling the evaluation of thousands of configurations in seconds. AI

Researchers have developed XPR, a new framework designed to simplify the creation and deployment of point-based differentiable renderers. This framework allows developers to implement new rendering methods with minimal code by separating method-specific logic from the core rendering pipeline. XPR's modular design enables it to compile and run on various hardware accelerators, including GPUs, TPUs, and CPUs, facilitating faster experimentation and cross-platform compatibility for graphics and AI applications. AI

OpenAI has partnered with Oracle to offer access to its models, including Codex, through Oracle Cloud Infrastructure. This collaboration allows businesses to leverage OpenAI's AI capabilities while utilizing their existing Oracle cloud commitments. The integration aims to provide enterprise-grade security and governance for AI development and deployment. AI

Alibaba Cloud has released Meoo CLI, an open-source command-line tool designed to streamline the deployment of local AI programming projects. This tool allows local AI coding assistants like Claude Code and Cursor to leverage cloud capabilities for tasks such as database integration, user authentication, file storage, and final project release, enabling a smoother transition from local development to live deployment. AI

Alibaba Cloud's Meoo (Miaowu) has released Meoo CLI, an open-source command-line tool. This tool enables local AI programming assistants like Claude Code, Codex, and Cursor to leverage cloud capabilities. These assistants can now perform tasks such as database access, user authentication, file storage, and project deployment, streamlining the process of moving local projects to a live online state. AI

Frugal computing is an architectural approach that prioritizes minimizing computational resource usage, starting with the question of whether a computation, data transfer, or model call is necessary at all. This concept, primarily based on Wim Vanderbauwhede's work, advocates for achieving the same useful outcome with less energy, material, and overhead. The growing demand on data centers, particularly for AI, and increasing regulatory scrutiny in regions like Europe are driving the adoption of frugal computing principles into system design. AI

This article details how to implement native hybrid search within PostgreSQL using the pgvector extension and Spring AI. It advocates for consolidating search functionalities into a single database, eliminating the need for separate Elasticsearch clusters and the associated synchronization issues. The approach involves storing both dense and sparse vector embeddings in PostgreSQL and performing hybrid queries with Reciprocal Rank Fusion (RRF) directly within the database. AI

DocLang is a new open document standard aiming to simplify how artificial intelligence systems process and understand various document formats. Developed under the umbrella of the Linux Foundation, this initiative seeks to create a unified approach for AI to interact with information, potentially reducing the complexity and improving the efficiency of AI-driven document analysis. AI

SPACE AI is developing an integrated ecosystem for the construction industry, aiming to connect disparate tools with AI-powered intelligence. Their platform, SPACE AI suite, is designed to unify various aspects of construction operations, including planning, scheduling, field operations, workforce management, customer engagement, and analytics. The goal is to simplify complexity, enhance visibility, and enable data-driven decision-making for construction organizations. AI

A new open-source project called Burn Baby Burn has been released, designed to help manage and potentially reduce the cost of AI token usage. The project, available on GitHub, is presented as a tool for understanding and controlling expenses related to large language model interactions. AI

A developer has created a self-hosted LLM stack designed for enterprise use, addressing the common challenges of deploying AI models beyond the demo phase. The stack prioritizes data security by keeping all information, including audit logs, on-premises. It also implements per-user authentication for access control and includes an automated acceptance testing framework to objectively evaluate model performance before deployment. AI

LG Electronics is developing a blockchain-based advertising platform using Arbitrum's layer-2 technology. This new network aims to create a shared database for ad inventory and track customer interactions, streamlining the ad sales process. The company is evaluating the platform's market viability and potential value for advertisers, publishers, and audiences. AI

A new pattern for enterprise AI search, termed the Security-Trimmed Index (STI), proposes normalizing data from disparate sources into a single Azure AI Search index. Each document within this index would include an Access Control List (ACL) field populated during the ingestion process. Crucially, access control is enforced at query time by injecting the user's identity into the search query, ensuring that results are filtered based on authorization before being returned to the application. AI

Taobao Flash Purchase has launched a new "New Store Growth System" specifically for catering businesses to help small and medium-sized merchants with their digital transformation. This initiative provides a clear growth path, enhanced traffic benefits, and AI-driven operational tools to overcome initial challenges and foster steady growth. The system breaks down complex operations into manageable tasks, offering rewards and guidance, while AI assistants help with tasks like store setup and marketing, aiming to shift the focus from simple subsidies to empowering merchants with capabilities. AI

A new tool called RiskKernel has been developed to address a critical issue in long-running AI agents: the loss of budget and state when the agent crashes or is interrupted. Unlike existing solutions that only checkpoint the agent's task context, RiskKernel durably stores the entire enforcement envelope, including budget spent, loop counts, and time elapsed, in a SQLite database. This ensures that if an agent restarts after a crash, it resumes with the same budget and constraints, preventing accidental overspending and maintaining the integrity of the original task limits. AI

This guide details how to set up the Codex CLI to interact with a local LLM, specifically Gemma-4, using llama.cpp on Windows Subsystem for Linux (WSL2). The process involves installing Codex, configuring it to use llama.cpp as a model provider, and then running the llama.cpp server with the Gemma-4 model. The author shares specific commands and configuration file examples, including troubleshooting a context size error. AI

Coinbase has introduced a new agent capable of executing trades and purchasing premium research through the X402 payment protocol. Initially supporting crypto spot markets and derivatives, the agent is slated to expand its capabilities to include equities and prediction markets. This development is part of Coinbase's ongoing commitment to integrating AI tools, following their prior work on AgentKit and an AI assistant. AI

Training large language models with extensive context windows, such as 3 million tokens, faces memory limitations on hardware like 8xH100 nodes. Researchers have developed a method called Untied Ulysses to overcome these constraints, enabling the training of models at 8B and 32B scales with significantly longer sequences than previously possible. AI

Ollama Cloud offers a managed inference service for open-source large language models, allowing users to run models on Ollama's GPUs without local hardware. The service has three tiers: Free, Pro ($20/month), and Max ($100/month), with usage measured by GPU time rather than tokens. The Free tier is suitable for experimentation with lighter models, Pro is recommended for daily engineering work with higher concurrency, and Max is designed for production workloads requiring sustained concurrent access to the most powerful models. AI

Researchers have developed CompRank, a new framework designed to make large language model (LLM) rerankers more computationally efficient for information retrieval tasks. CompRank achieves this by reducing redundant computations through token-level compression and a decoding-free scoring method. Experiments demonstrate that CompRank significantly speeds up reranking while maintaining high performance, making LLM-based reranking more scalable for long candidate lists. AI

Guestlist is a new tool designed to help users determine if AI agents can successfully access a given URL. The service continuously probes the web using real browsers to assess domain accessibility, assigning a 'green to red' grade. This allows users to check a URL's viability via an API call before making a request, thereby saving resources by avoiding inaccessible sites. AI

A developer has found a way to significantly reduce the cost of using Anthropic's Claude Code CLI tool by routing requests through APIVAI, a third-party API gateway. This method allows users to access the same Claude models at a fraction of the direct Anthropic API price, requiring only a change in environment variables. Separately, another developer is seeking advice on improving their custom API's documentation and design to prevent AI agents like Claude from losing context or memory of its capabilities during interactions. AI

Researchers at the University of Twente have developed a method to reduce the energy consumption of training large language models by up to 14%. The technique, known as dynamic voltage-frequency scaling (DVFS), involves intelligently adjusting the clock frequencies of a GPU's computational core and memory. By fine-tuning these frequencies on a per-kernel basis, the researchers achieved significant energy savings without compromising training speed. AI

Researchers have developed Deepcontour, a new framework that uses deep learning to optimize the construction of integration contours for solving large-scale Generalized Eigenvalue Problems (GEPs). This method employs a deep learning-based spectral predictor and Kernel Density Estimation to automatically design efficient contours, leading to significant speedups. The framework achieved up to a 5.63x performance increase on various scientific datasets while maintaining numerical accuracy. AI

Researchers have developed SPARX, a framework for accelerating Convolutional Neural Networks (CNNs) on edge devices. This system integrates approximate computing with security and privacy features within a RISC-V System-on-Chip. SPARX utilizes a custom RISC-V instruction extension and an approximate logarithmic CNN accelerator, enhanced by a differential-noise privacy engine and authentication mechanisms. Evaluations show significant reductions in area and power, alongside improved throughput, with a minimal impact on accuracy for specific CNN models. AI

A new arXiv paper investigates the paradox where smaller block sizes in LLM quantization can degrade model quality. Researchers found this is not an inherent limitation but stems from how statistical clustering interacts with scaling factors. The study proposes solutions like preventing scaling factor underflow and using targeted heuristics such as the 4-over-6 methodology to improve quality, emphasizing the need for tight coupling between hardware and software design for next-generation ML accelerators. AI

Researchers have introduced Sigma-Branch (SigmaB), a novel framework designed to optimize deep neural networks for memory-constrained edge devices. SigmaB restructures dense networks into a hierarchical tree with shared backbones, routers, and specialized leaves, enabling dynamic inference. This approach significantly reduces the number of active parameters per inference by executing only a single root-to-leaf path, thereby minimizing off-chip weight transfers without sacrificing overall model capacity. AI

Anyscale's Ray Day London event highlighted how organizations are scaling AI workloads using the Ray framework. Key presentations included Xoople's use of Ray Data for global-scale geospatial foundation model inference and Adyen's training of a Transaction Foundation Model (TFM) on a massive 51 trillion token dataset. These case studies demonstrated Ray's ability to simplify complex AI infrastructure for tasks ranging from multimodal data processing to foundation model training and reinforcement learning. AI

This blog post details how to profile PyTorch code, focusing on the `nn.Linear` module and its underlying operations. It explains that `nn.Linear` wraps matrix multiplication and addition, and that PyTorch optimizes this by transposing weights on the CPU and folding the bias addition into the matrix multiplication kernel via an epilogue. The post uses an NVIDIA A100 GPU and Hugging Face infrastructure to demonstrate profiling traces. AI

Netflix engineer has developed and open-sourced a tool called Headroom designed to significantly reduce the cost of running AI models. This project aims to provide substantial savings for users by optimizing AI computational expenses. The tool is now publicly available, allowing anyone to benefit from its cost-saving capabilities. AI

Researchers have developed IntentKV, a novel method for pruning KV caches in large language model agents to improve inference efficiency. This technique maintains a session-level memory of cross-turn intent, allowing it to score and selectively drop tokens without significant accuracy loss. IntentKV has demonstrated substantial reductions in peak request tokens and KV reads, particularly for long-horizon agent tasks, while keeping the base LLM unchanged. AI

Researchers have developed EPIC, a novel method for constructing preference-aligned memory for on-device Retrieval-Augmented Generation (RAG) systems. This approach significantly reduces memory usage by prioritizing preference-relevant information, achieving a 2,404x reduction in indexing memory. EPIC also enhances preference-following accuracy by 18.79 percentage points and drastically lowers retrieval latency, making it suitable for resource-constrained personal AI agents. AI

Researchers have released ASTRA-sim 3.0, an updated open-source simulator designed for distributed machine learning. The new version enhances simulation fidelity by modeling GPU execution and infrastructure at a fine-grained, cache-line level. It also introduces InfraGraph, a standardized representation for network infrastructure, enabling more detailed design space exploration for collective algorithms and hardware architectures. AI

Researchers have developed a new method using Mojo to accelerate AI efficiency in finance, particularly for high-frequency trading and time series analysis. Their Mojo SIMD k-d tree implementation offers significant speedups over existing libraries like scikit-learn, achieving up to 43.5x faster performance on ARM64 architectures. This advancement allows financial AI models to process larger datasets in real-time, improving accuracy in areas like derivative pricing and enabling training on ten times more data. AI

Researchers have developed a cost-aware routing system for text-to-image generation that dynamically adjusts computational resources based on prompt complexity. This framework routes each prompt to the most suitable generation function, which could involve varying the number of denoising steps in a diffusion model or selecting an entirely different model. By learning to reserve intensive computations for complex prompts and using more economical options for simpler ones, the system aims to optimize the trade-off between image quality and computational cost. Experiments on COCO and DiffusionDB datasets showed that this routing approach, utilizing nine pre-trained models, achieved higher average quality than any single model could alone. AI

Tencent has developed HiGR, a hierarchical generative framework for industrial-scale slate recommendation. This system addresses challenges in applying generative models to large-scale recommendation by learning structured item IDs and shifting autoregressive modeling to preference embeddings for efficient planning. HiGR has demonstrated significant improvements in offline recommendation quality and inference speed, and has been successfully deployed on Tencent platforms, enhancing user engagement metrics. AI

Researchers have developed torch-sla, an open-source Python library designed to provide differentiable sparse linear algebra capabilities within PyTorch. This library addresses a gap in PyTorch's existing functionalities, which currently offer only low-level kernels or CPU-only, non-differentiable solvers. Torch-sla supports a unified API for various solver types across multiple backends, including CPU and GPU options, and enables distributed execution for enhanced scalability. AI

This article provides a practical guide for AI engineering teams on creating and managing a shared "skills library." The aim is to ensure consistency and efficiency by standardizing the design, versioning, and distribution of AI skills that team members can utilize. This approach is particularly beneficial for tools like Claude Code and Cursor, enabling all engineers to work from a common foundation. AI

Anyscale has demonstrated significant cost savings in LLM serving by disaggregating the prefill and decode phases of inference. This approach separates prompt processing onto dedicated GPUs from token generation, reducing interference and improving throughput. While this method can lead to up to 67% cost reduction and 2.3x more queries per second, it introduces operational complexity and can slightly increase time-to-first-token. AI

A developer has created GraphPilot, a tool designed to enhance coding agents by providing them with persistent structural memory. This tool indexes a TypeScript/JavaScript repository once, allowing agents to query its structure instead of re-reading files, which significantly reduces token usage and improves accuracy. In testing, GraphPilot cut token usage by 61% and improved accuracy on specific query types like "who calls X?" and impact analysis, though it had less impact on flow-tracing questions. AI

A user on the r/LocalLLaMA subreddit shared their experience of saving money by running AI models locally. Over two days, they processed 50 million tokens across 49 coding sessions, estimating a cost of $151.40 if they had used Anthropic's Claude Sonnet model. The user detailed their cost calculation, highlighting the significant input token usage and comparing it to the potential expense of cloud-based AI services. AI

Exposing tools to AI models, such as in MCP servers, incurs a significant token cost with each API call. This cost arises because the tool's name, description, and JSON schema are sent to the model's context repeatedly. A larger number of tools not only increases this token bill but also negatively impacts the model's accuracy in selecting the correct tool. To address this, a new CLI tool has been developed to make these hidden costs visible to developers. AI

Netflix engineer has developed an open-source project called Headroom, designed to significantly reduce the cost of running AI models. This tool aims to optimize AI inference, potentially saving users substantial amounts of money. The project has been made publicly available, allowing others to benefit from its cost-saving capabilities. AI

Researchers at New York University have created a new method for compressing the input context of large language models, reducing it by up to 16 times without sacrificing accuracy. This technique allows for significantly faster processing speeds using existing infrastructure. AI

BuyWhere has released a tool that allows AI agents to access real-time product pricing from over 15 Singaporean merchants. The tool, which integrates with platforms like LangChain and CrewAI, uses the Model Context Protocol (MCP) to connect to retailers such as FairPrice, Cold Storage, Lazada, and Shopee. Developers can integrate this functionality into their AI agents with a simple command and a free API key. AI

An AI operations agent has identified seven common cost-saving opportunities in production LangGraph agents. These leaks, found through auditing agent stacks, can significantly inflate AI bills. The agent provides specific detection methods and fixes for issues like excessive context in prompts, using expensive models for simple tasks, and inefficient retry logic that incurs unnecessary costs. AI

The author details a migration from x402 v1 to v2, noting that v2 is a significant departure rather than a simple upgrade. Key changes include a shift to the @x402 npm scope, the introduction of CAIP-2 for networks, and the relocation of payment challenges from the JSON body to the PAYMENT-REQUIRED header. The article also highlights new client-side scheme handling and the integration of Bazaar discovery for paid routes. AI

Trajectory Labs has demonstrated frontier model performance on an open-source model, achieving this feat in under 24 hours of post-training. This achievement highlights the potential of combining strong open models with efficient training infrastructure. Together Compute provided the necessary computing power for this rapid development, in collaboration with Nvidia. AI

Ollama version 0.30 has been released, significantly boosting local inference speeds for Qwen models on NVIDIA GPUs. This update enhances support for Vulkan and NVIDIA hardware, improves GGUF compatibility, and streamlines the local GPU inference process. The release enables faster, privacy-focused desktop chat applications and GPU-accelerated research by providing a more efficient backend for large language models. AI