Brief

An attempt to optimize LLM costs by batching multiple text segments into single API calls backfired, significantly increasing expenses and slowing down processing. The issue stemmed from the LLM failing to consistently return all required IDs in its JSON output, triggering a fallback mechanism that retried entire batches. This led to a substantial increase in API calls due to retries, negating the intended cost savings. AI

LuisCore has launched as a decentralized runtime infrastructure designed for multi-step AI agents, focusing on action pipelines and inference rather than individual agent capabilities. It aims to provide a shared vocabulary and substrate for agents built with various frameworks, enabling them to interoperate without significant rewriting. The platform emphasizes open-source components, machine-readable discovery, and real-time telemetry for agent coordination and communication. AI

A user on Reddit is seeking the most cost-effective way to rent H200 GPUs for Stable Diffusion video generation. They are encountering VRAM limitations with their current setup, impacting workflow and quality. The user is looking for reliable providers offering H200 rentals for short-term use at a lower price point than major services, prioritizing VRAM capacity and stable network connections for sustained rendering tasks. AI

Google is updating its data retention policies to include images from Lens, recordings from Search Live, and audio from Translate for AI training. Users will have a new "Search Services History" setting to manage this data, separate from the existing Web & App Activity. While this data will help Google develop and improve its services, including AI models, users can opt out to prevent their media from being saved and used for training. AI

This article details how to integrate real-time data streaming into MCP Apps using WebSockets, moving beyond traditional polling methods. By declaring `connectedDomains` in the app's Content Security Policy, developers can enable direct WebSocket connections from the sandboxed iframe to a backend server. A lightweight Python WebSocket server is then implemented to push live updates for dashboards, KPIs, and transaction feeds, bypassing the need for the host to relay data and reducing latency. AI

Google's Gemini AI experienced a significant outage lasting over seven hours, affecting users globally. The disruption was characterized by an inability to respond, with error codes 1076 and 1099 being reported. The incident impacted users across various regions, disrupting access to the AI service. AI

Datadog has introduced new features including Bring Your Own Cloud (BYOC) support, federated logs search, and integration with third-party SIEM systems. Despite these advancements, an analyst has cautioned about the potential for vendor lock-in. The update also highlights new agentic AI security tools and discusses the complex cost structures associated with AI. AI

A developer created an open-source tool called ccglass to monitor API calls made by coding agents, revealing significant cost-saving opportunities. The tool acts as a local reverse proxy, logging requests to services like Anthropic's Claude Code and OpenAI. Analysis showed that by optimizing prompts and understanding per-task costs across different providers, the developer reduced their monthly bill by 35% and improved efficiency. AI

A developer detailed how an AI agent's hidden successful retries led to an unexpected $4,200 cost increase. The agent's system retried deterministic validation failures multiple times before succeeding, masking the issue on dashboards that only track final success rates. The author suggests implementing a `cost_per_successful_chain` metric and a local repair stage for deterministic errors to prevent such costly, silent failures. AI

Researchers have developed a new framework to accelerate Birkhoff projection, a crucial step in manifold-constrained hyper-connections (mHCs). This method reduces the projection problem to a three-dimensional unconstrained convex problem solvable with Newton's method, leading to faster convergence and higher accuracy. The approach also employs implicit differentiation for exact gradients and a warp-level CUDA kernel for significant parallelization, achieving over 20x acceleration in experiments. AI

Researchers have developed Semantic Cache Distillation (SCD), a new framework designed to reduce the communication bottleneck in disaggregated LLM inference. SCD replaces raw Key-Value (KV) cache transmission with compact semantic codes, improving the time-to-first-token (TTFT) by up to 2.65 times. The method utilizes reuse and selective patching to minimize transfer costs and truncate error propagation, maintaining generation quality close to the oracle. AI

Researchers have developed a novel defense system called "model multiplicity" to detect adversarial attacks during the training of small language models on edge devices. This approach involves training multiple language models concurrently, each using different subsets of edge nodes. By monitoring the divergence between these models, the system can identify and isolate compromised nodes that are attempting to poison the training data. Evaluations show this method is more effective than traditional single-model defenses in detecting such attacks in distributed learning environments. AI

Researchers have developed Larch, a new framework designed to optimize the execution of semantic filters within AI SQL queries. Larch addresses the high inference costs and latencies associated with semantic operators, which treat AI-generated filters as black boxes, hindering traditional optimization. The framework utilizes embedding-augmented neural networks and supervised learning models to predict filter selectivities and determine optimal evaluation orders, significantly reducing token usage. AI

Researchers have developed Harmonia, a new framework designed to optimize the serving of Retrieval-Augmented Generation (RAG) pipelines. This system addresses the complexities of RAG by enabling flexible workflow composition, intelligent deployment across diverse components, and a runtime controller for load balancing and auto-scaling. In evaluations across four RAG applications, Harmonia demonstrated significant improvements, achieving over double the throughput and substantially reducing service level objective violations compared to commercial alternatives. AI

Researchers have introduced the Swarm Policy Interference Network (SPIN) framework to address challenges in decentralized swarm coordination on edge devices. SPIN models swarm topologies using compressed tensor networks, specifically factorizing joint policy tensors into Matrix Product State chains. This approach reduces computational complexity and communication overhead, enabling efficient runtime adaptation through a hybrid neuro-symbolic control pipeline. The framework has been validated in simulations for tasks including tracking, dispersion, and multi-goal coordination. AI

Researchers have developed a new framework for quantum compilation that integrates hardware awareness with data-driven error detection. This approach aims to improve the success rates of algorithms on early fault-tolerant quantum systems by jointly optimizing qubit mapping, SWAP insertion, and syndrome scheduling. Simulations show a significant increase in algorithmic success probability compared to existing methods, particularly for benchmarks like VQE. AI

Researchers have developed BlendServe, a new system designed to optimize offline inference for auto-regressive large language models. BlendServe combines resource overlapping and prefix sharing techniques to maximize throughput and reduce costs for latency-insensitive applications. Evaluations show that BlendServe can achieve up to a 1.44x throughput increase compared to existing standards like vLLM and SGLang. AI

Researchers have developed a new algorithm called Q-FLAIR to reduce the computational resources needed for quantum machine learning feature maps. This method shifts significant workloads to classical computers, enabling the training of complex quantum models with fewer evaluations. Q-FLAIR has demonstrated state-of-the-art performance on classifiers and achieved over 90% accuracy on the MNIST dataset using a real IBM quantum device in just four hours, a feat previously considered unattainable due to hardware demands. AI

Researchers have developed a new framework called Streaming Harness to enable Vision-Language Models (VLMs) to process unbounded video streams in real-time. This system enhances VLMs with proactive interaction, long-term memory retention up to 12 hours, and sub-second processing latency. To support this advancement, they also introduced a new streaming dataset, Streaming-Train-248K, and a benchmark, Streaming-Eval, to drive further progress in deployable streaming intelligence. AI

Researchers have developed "Resilient Write," a six-layer system designed to improve the reliability of LLM coding agents. This system addresses failures in writing files by implementing features like risk scoring, atomic writes, and error handling. The goal is to reduce agent retry times and enhance self-correction capabilities, with a 5x reduction in recovery time and a 13x improvement in self-correction rate demonstrated in tests. AI

A new framework called OPTIKIT has been developed to automate the process of optimizing large language models for enterprise use. This tool aims to democratize model compression and tuning, enabling teams without specialized expertise to improve LLM performance. In production environments, OPTIKIT has demonstrated over a 2x increase in GPU throughput, allowing application teams to achieve better performance without needing deep optimization knowledge. The system's design and engineering insights, particularly in resource management and pipeline orchestration, are being open-sourced to encourage broader reproducibility and contributions. AI

Researchers have developed a method to improve the efficiency of multi-GPU machine learning training by overlapping computation and communication phases. The technique uses shared-memory allocation to manage computation kernel residency, ensuring enough on-chip resources are available for communication kernels. By assigning higher priority to communication streams, the approach effectively reduces total execution time by up to 25.5 percent across various NVIDIA and AMD GPUs without altering vendor libraries. AI

Researchers have developed SNN-MLIR, a new MLIR dialect designed to compile spiking neural networks (SNNs) from a common intermediate representation (NIR) into C code for bare-metal deployment. This tool addresses the fragmentation of SNN training frameworks by providing a unified compiler representation that supports both floating-point and quantized data. The system includes a Python frontend to read NIR files and a lowering pass that generates self-contained C11 code, currently supporting feedforward, fully-connected networks on CPU targets. AI

A new survey paper explores the use of large language models (LLMs) and agentic systems for automating the generation and optimization of GPU kernels. These kernels are crucial for the performance of AI systems, but their manual creation is a time-consuming and non-scalable process. The paper aims to provide a structured overview of current LLM-driven approaches, datasets, and benchmarks, while also outlining future research directions in this rapidly evolving field. AI

Researchers have developed a novel two-phase method for semantic filtering in large document corpora, aiming to improve efficiency and accuracy. This adaptive approach combines model-free clustering with token-aware proxy models, outperforming previous methods by 1.6-2.0x at a 90% accuracy target. The system leverages the oracle's per-document confidence for training and difficulty assessment, indicating significant potential for future optimization. AI

Researchers have developed a new framework called Mathematics of Arrays (MoA) to optimize transformer kernels, which are computationally intensive components of modern AI models. This framework uses algebraic construction to eliminate intermediate arrays, significantly reducing memory traffic and energy consumption compared to standard implementations. The MoA approach promises substantial speedups and energy reductions, with potential applications for DARPA and DOE initiatives. AI

Researchers have developed MemoVAD, a novel framework for resource-efficient video anomaly detection on edge devices. This system uses a combination of edge and cloud processing, with a unique uncertainty-aware gating policy that only sends high-uncertainty clips to a cloud-based Vision-Language Model. A dynamic semantic memory stores VLM-verified prototypes, allowing the edge model to progressively learn richer semantics and significantly reduce communication overhead while maintaining high performance. AI

Researchers have developed Industrial-YOLO, a framework using a fine-tuned YOLOv8 model for real-time defect detection on edge hardware. This system was benchmarked on the NEU surface defect database and MVTec AD, with added automotive manufacturing extensions. The framework achieves over 120 FPS on an NVIDIA Jetson Orin platform with a 98.5% mAP, demonstrating robust, zero-latency performance suitable for automated optical inspection systems. AI

Researchers have developed AgentCompile, a novel compiler that leverages Large Language Models (LLMs) to optimize transformer inference for CUDA. AgentCompile uses LLM outputs as advisory metadata to guide decisions on specialization and CUDA implementation choices. This approach has demonstrated significant speedups, achieving an average of 5.66x, 4.05x, and 4.26x faster inference over PyTorch eager for Qwen3-1.7B, Qwen3-4B, and Llama-3.2-1B-Instruct models, respectively. AI

Researchers have developed an embedded graph convolutional network (EFGCN) specifically designed for real-time event data processing on System-on-Chip (SoC) FPGAs. This approach significantly reduces model size, by up to 100-fold compared to previous methods, while maintaining competitive accuracy on classification tasks. The EFGCN achieves high throughput and low latency, making it suitable for embedded systems, particularly in the automotive sector. AI

A new tutorial paper explores the integration of blockchain infrastructure with embodied AI systems, focusing on post-quantum security and trustworthy data economies. It highlights the need for crypto-agile architectures to protect data provenance and governance as quantum computing advances threaten current cryptographic primitives. The paper proposes blockchain as a foundational layer for decentralized intelligent environments, offering open-source frameworks for quantum-resistant, interoperable, and data-trustworthy systems. AI

A new survey paper proposes an AI-native, closed-loop security framework for 6G-enabled cyber-physical systems (CPSs). The proposed system aims to detect and mitigate threats at the network edge with millisecond-level precision, addressing the limitations of traditional security models. It integrates various AI techniques, including federated learning and digital twins, to create a robust and adaptive security pipeline. AI

Researchers have developed OmniMem, a new framework designed to make audio-visual large language models more memory-efficient for processing long videos. OmniMem addresses the challenge of linearly growing video tokens and KV caches by employing a modality-aware allocation strategy that distinguishes between visual and audio contexts. It also uses perturbation-aware selection to retain crucial information, preventing memory compression from degrading understanding. Experiments show OmniMem improves accuracy by 2-4% over existing methods under similar memory constraints, with further gains possible through budget-aware fine-tuning. AI

Researchers have developed a new method called C$^3$ache to speed up the inference process for World Action Models (WAMs). WAMs are known for their strong generalization capabilities in robotics but are computationally expensive due to a multi-step denoising process. C$^3$ache addresses this by caching and reusing computation residuals across different inference chunks, achieving up to a 2.5x speedup without significantly impacting task success rates. AI

Researchers have developed a new framework for modeling extremely long user behavior sequences in short-form video recommendation systems. The system uses content-native Semantic IDs instead of traditional item IDs to reduce embedding table size and improve generalization to new content. Additionally, a Global-Aware Compression Transformer condenses user sequences, significantly lowering memory and computational requirements. AI

Researchers have developed a new framework called Cooperative Autodidactic NeuroSurgeon (CANS) to improve the efficiency of collaborative deep neural network inference on mobile edge devices. CANS allows devices to adaptively learn optimal model partitions by sharing feedback during inference, addressing challenges posed by fluctuating network conditions and diverse device capabilities. The framework incorporates a FedLinUCB-DW algorithm for device grouping and leverages offline experience for faster exploration, with theoretical guarantees on its performance. In prototype experiments, CANS demonstrated a significant reduction in inference latency, cutting it by up to 50% compared to non-cooperative methods. AI

A new method called SCORE (Sequential Cyclic Optimization via Refinement & Evaluation) has been developed for optimizing the placement of ground stations for low-Earth orbit satellite constellations. This method operates on a continuous spatial domain, allowing for more flexible and potentially higher-throughput configurations compared to traditional fixed-site approaches. Tests on commercial and synthetic constellations demonstrated that SCORE can improve downlink throughput by up to 13% with significantly fewer function evaluations than existing methods, while also exploring trade-offs between new infrastructure and existing sites. AI

Feature stores are crucial for enterprise AI, acting as a central hub for managing data features essential for model training and inference. A new guide offers a walkthrough for constructing a basic, functional feature store implementation. AI

A user on Reddit's r/LocalLLaMA subreddit has demonstrated that large language models can be run on systems with very limited RAM and no dedicated GPU. The user tested models like Gemma 4 12B and StepFun Flash 3.7 198B MoE on a laptop with only 2.6 GiB of free RAM. The results showed that even with these constraints, the models were capable of processing prompts and generating responses, suggesting broader accessibility for running LLMs on consumer hardware. AI

Developers building AI applications should move beyond single-model prototyping to a workflow-centric approach for production. Different workflows, such as support chat, document Q&A, or content generation, have distinct requirements for model behavior like latency, reasoning, or structured output. Evaluating and selecting models based on their performance within specific workflows, rather than general popularity, is crucial for optimizing AI products. Platforms like VectorNode aim to facilitate this by offering unified access to various models through a single API. AI

A team successfully reduced their AI agent's operational costs by 60% through several optimization strategies without compromising quality. Key improvements included context engineering techniques like an append-only status header and context compaction, which prevented redundant processing of conversation history. They also implemented tiered model routing, directing tasks to more cost-effective models based on complexity, and utilized local models for private, high-frequency tasks to reduce API latency and costs. AI

This tutorial demonstrates how to build a code dataset pipeline using metadata from NVIDIA's Nemotron-Pretraining-Code-v3 dataset. Instead of downloading the entire dataset, the process involves streaming the metadata, inspecting its schema, and creating a manageable sample for analysis. The tutorial details steps for reconstructing raw GitHub URLs, fetching source files, and estimating token counts, ultimately producing a reusable filtered sample for further experimentation. AI

Implementing task-type routing for LLMs can significantly reduce costs, potentially by 40-60%, without compromising quality. This approach categorizes tasks into simple, code, reasoning, and complex, directing each to the most cost-effective model tier. The overhead of the classifier is minimal, typically milliseconds, compared to the longer processing times of LLM calls. This strategy is particularly effective for workloads with a high proportion of simple tasks, where the price difference between small and frontier models is most pronounced. AI

A new terminal monitoring tool called xdna-top has been released to help users visualize the activity of NPUs and iGPUs on AMD's Strix Halo processors. This tool addresses the current difficulty in tracking NPU performance, as existing tools like amd-smi are not fully functional on this hardware. xdna-top provides a unified view of both the iGPU and NPU, displaying real-time activity and submission counters to offer a clearer picture of their utilization. AI

Amazon Web Services has introduced a new AI-powered assistant designed to streamline equipment repair processes. This tool, built using Amazon Bedrock AgentCore, helps technicians and farmers diagnose issues, identify necessary parts, and access repair documentation through natural language queries. The system integrates various AWS services, including a knowledge base for RAG, memory for conversation persistence, and authentication via Amazon Cognito, to provide a comprehensive solution for reducing downtime and repair costs. AI

GigaDevice has launched two new microcontrollers, the GD32E512 and GD32E252 series, specifically designed for optical modules. These new MCUs aim to support a range of optical module applications, from traditional low-speed to next-generation high-speed, providing hardware support for AI computing centers and advanced network infrastructure. This release expands GigaDevice's product offerings in the optical communication sector. AI

A technical blog post details how to significantly increase the inference speed of the Qwen3.6-27B large language model on a single RTX 3090 GPU. By optimizing the inference engine, using a smaller model quantization, and implementing multi-token prediction (MTP) with speculative decoding, the throughput was boosted from 35.7 tokens/second to 80.2 tokens/second, a 2.25x improvement. The author found that MTP alone provided a 1.78x speedup, while the other optimizations contributed to the remaining gains. The post also notes specific technical hurdles encountered, such as compatibility issues with Ollama's GGUF format and the optimal settings for MTP. AI

Oracle's AI Vector database is designed for distributed storage with GPU optimizations, enabling high performance on large-scale queries. In contrast, Chroma offers a lightweight, easily extensible architecture on Kubernetes, integrating well with open-source tools like LangChain. The choice between them hinges on data volume, infrastructure budget, and the need for integration with Oracle's cloud services. AI

A Reddit user shared their experience running DiffusionGemma 26B on a setup of four AMD 7900 XTX GPUs. They achieved generation speeds of up to 100 tokens per second, with an overall throughput of 45-60 tokens per second when accounting for prompt processing. The user detailed the extensive Docker command used to configure the vLLM environment for this specific hardware, noting that preparing the image consumed a significant amount of DeepSeek-V4-Pro tokens. AI

A developer has created a data marketplace called CDP Bazaar, accessible via the x402 protocol, designed to serve AI agents with verifiable facts. The platform collects data from various sources, stamps it with provenance information, and makes it available for purchase on a pay-per-record basis. This system aims to solve the problem of scattered public data by providing a centralized and citable source for AI agents. AI