ONNX Runtime
PulseAugur coverage of ONNX Runtime — every cluster mentioning ONNX Runtime across labs, papers, and developer communities, ranked by signal.
7 day(s) with sentiment data
-
TormentNexus combines Go and TypeScript in a modular monolith for AI
TormentNexus has adopted a modular monolith architecture, combining Go for backend performance and TypeScript for AI capabilities, to achieve sub-10ms response times. This approach avoids the latency and complexity of m…
-
Ragleap launches RAG library with focus on narrow scope
Ragleap has launched its RAG library, emphasizing a deliberate focus on narrow scope rather than broad feature parity. The library prioritizes retrieval-augmented generation, explicitly excluding agentic tool-calling an…
-
Offline Dictation Apps Emerge as Cloud Dependence Declines
The author details how they built an offline dictation application, DictaFlow, that runs speech recognition models locally on devices like Windows laptops, Macs, and iPhones. This contrasts with services like Wispr Flow…
-
SQLite extension sqlite-vec offers dependency-free AI memory stack
A new SQLite extension called sqlite-vec is enabling developers to build AI memory stacks with significantly reduced complexity and improved performance. This extension allows vector search capabilities to be directly i…
-
StrokeSeg2 framework simplifies clinical AI deployment
Researchers have developed StrokeSeg2, a lightweight and modular C++/Qt framework designed to make deep learning-based brain lesion segmentation more accessible in clinical research. The framework adapts resource-intens…
-
C++ RLHF reward scoring engine beats PyTorch on CPU
A new study investigates the speed of reward scoring in Reinforcement Learning from Human Feedback (RLHF) pipelines, finding that a custom C++ inference engine built on ONNX Runtime significantly outperforms standard Py…
-
New C++ engine speeds up RLHF reward scoring on CPUs
Researchers have developed a new C++ inference engine using ONNX Runtime for reward model scoring in Reinforcement Learning from Human Feedback (RLHF) pipelines. This engine was benchmarked against PyTorch eager mode, t…
-
Scaling Triton Inference Server with Kubernetes for Multi-GPU Workloads
This article provides a playbook for scaling the Triton Inference Server across multiple GPUs within a Kubernetes environment. It addresses the challenges of running multiple production models on a single GPU under heav…
-
Benchmarking edge inference frameworks for industrial machine vision · 2 sources tracked
A new research paper benchmarks the performance of four popular frameworks—PyTorch, ONNX Runtime, OpenVINO, and TensorRT—for deep learning inference on edge devices in industrial machine vision. The study found that Ope…
-
Run Neural Networks on GPU via Vulkan: Libraries, Compilers, or Custom Engines
This article outlines three methods for running trained neural networks on a GPU using the Vulkan API. It suggests integrating existing libraries like TensorFlow Lite or ONNX Runtime, compiling models via ML compilers s…
-
New Android app VisionAId assists visually impaired with on-device AI
Researchers have developed VisionAId, an Android application designed to assist individuals with visual impairments by transforming a standard smartphone into a real-time visual assistant. The system utilizes six on-dev…
-
Kuma project compiles PyTorch models for browser execution via WebGPU
A new project called Kuma aims to compile PyTorch models into self-contained WebGPU executables. This approach would allow models to run directly in the browser without needing Python or a server-side runtime. The proje…
-
PaddleOCR releases PP-OCRv6 with 50-language support on Hugging Face
PaddleOCR has released PP-OCRv6, an updated suite of universal OCR models available on Hugging Face. This new generation offers improved text detection and recognition accuracy, with models ranging from 1.5 million to 3…
-
Microsoft Presidio gains traction on GitHub with new features
Microsoft Presidio, an open-source tool for detecting and protecting sensitive information, has gained significant traction on GitHub, surpassing 9,390 stars. Recent updates to the project include the addition of an ONN…
-
AI LOD framework optimizes game animation with distance-aware model precision
Researchers have introduced a novel framework called AI Level of Detail (AI LOD) to optimize real-time human motion prediction in games. This approach dynamically adjusts the precision of machine learning models based o…
-
ONNX Runtime outperforms HF Transformers in CPU-only speech benchmark
A benchmark comparing ONNX Runtime, Hugging Face Transformers, and GGUF for the Parakeet TDT 0.6B model on CPU-only hardware revealed that ONNX Runtime achieved a 37% faster inference time than Hugging Face Transformers…
-
Python project enables local, GPU-accelerated AI background removal
A new open-source project, bg-vanish-mcp, has been released to enable AI assistants to perform background removal on images locally. This tool leverages Python, the DirectML API via ONNX Runtime, and U2NET models for GP…
-
Meta's EnCodec gets portable C++ implementation
A C++ implementation of Meta's EnCodec audio codec has been developed, aiming for portability and high performance without external machine learning runtimes. This project, available on GitHub, compiles model weights di…
-
Browser-based real-time voice changer released as MVP
A developer has created a real-time voice changer that operates entirely within a web browser. This tool leverages WebAssembly, ONNX Runtime, and WebGPU for its functionality. The creator has released it as a minimum vi…
-
New framework tackles industrial Edge AI deployment challenges
This paper introduces a new systems framework designed to improve the deployment of Edge AI applications on industrial embedded platforms. It argues that treating AI deployment as a systems problem, rather than just a m…