ONNX Runtime
PulseAugur coverage of ONNX Runtime — every cluster mentioning ONNX Runtime across labs, papers, and developer communities, ranked by signal.
8 day(s) with sentiment data
-
Open-source EOU detector released for English and 8 Indic languages
A new open-source End-of-Turn (EOU) detection model has been released, designed to prevent voice agents from cutting callers off mid-sentence. This model is particularly beneficial for languages like those in India, whe…
-
Microsoft Foundry Local enables offline Qwen3.8-27B coding assistant
Microsoft has released Foundry Local, enabling developers to run the Qwen3.8-27B language model locally for coding assistance. This setup integrates with GitHub Copilot, offering an offline, private coding assistant for…
-
Foundry Local brings agentic coding to devices with significant speedups
Foundry Local has been released, enabling agentic coding capabilities on local devices. This new tool reportedly offers significant performance improvements, with gains of 35-135% in decode and token generation speeds o…
-
Developer Rebuilds Karpathy's HackerTube as OpenTube, Measures Performance
A developer has recreated Andrej Karpathy's HackerTube project, naming their version OpenTube, to filter YouTube links from Hacker News comments. The project uses a two-stage model cascade, starting with a smaller model…
-
Enterprise LLM Gateways: Semantic Caching and Prompt Guardrails Detailed
This technical guide details the implementation of semantic caching and prompt guardrails for enterprise LLM gateways. It outlines an architecture that sits at the L7 proxy layer within Kubernetes clusters, processing i…
-
LLMs in the browser face complex technical hurdles
Running large language models (LLMs) directly within web browsers presents significant technical challenges due to the complexity of browser environments and evolving web standards. Developers must navigate various comp…
-
OmniMem 7 simplifies AI app with single Rust binary, no Docker needed
OmniMem 7 has been released as a single Rust binary, simplifying the application by eliminating the need for Docker, Python, or separate database servers. This new version integrates the MCP server, memory engine, embed…
-
Cloudflare and OpenAI explore computer vision integration in .NET
Cloudflare and OpenAI have collaborated on a research pilot, sharing early results on October 15th. The pilot focused on integrating computer vision capabilities within .NET applications, exploring various techniques su…
-
Social Sciences DataLab releases fast Indra OCR model
Social Sciences DataLab has released Indra OCR, a new 1-million-parameter OCR model designed for high throughput and speed on CPUs. It can process up to 49 pages per second, significantly outperforming Tesseract and Pad…
-
VEKTOR 2.0 launches with focus on local data and privacy
VEKTOR 2.0 has been released, offering a private, local memory and assistant layer that runs on a user's computer. The software is written in JavaScript and emphasizes privacy-enhancing technology and sovereign local da…
-
New methods optimize 3D Gaussian head avatars for quality and edge deployment
Two new research papers propose advancements in 3D Gaussian head avatar modeling. The first introduces a counterfactual route optimization framework to dynamically adjust training objectives for better geometry, appeara…
-
Serverless TTS system slashes costs with billing-aware inference tuning
Researchers have developed a novel billing-aware neural text-to-speech (TTS) serving system optimized for serverless CPU architectures. This system addresses the cost implications of idle inference state on instance-bil…
-
New 'woma' foundation model sets real-time endoscopy standard
Researchers have developed "woma," a real-time foundation model for gastrointestinal endoscopy, trained without labels on approximately one million endoscopy frames. This model can be fine-tuned for specific tasks, such…
-
Running multiple AI models on one GPU: challenges and considerations
This article explores the challenges of running multiple AI models on a single GPU, focusing on potential conflicts and performance degradation. It highlights key questions to consider regarding model interference, syst…
-
LLM inference servers share GPU memory to serve hundreds of users
Serving hundreds of users simultaneously with a single GPU for large language models is achieved by loading the model weights into GPU memory once and sharing them across all requests. The inference server manages this …
-
TormentNexus combines Go and TypeScript in a modular monolith for AI
TormentNexus has adopted a modular monolith architecture, combining Go for backend performance and TypeScript for AI capabilities, to achieve sub-10ms response times. This approach avoids the latency and complexity of m…
-
Ragleap launches RAG library with focus on narrow scope
Ragleap has launched its RAG library, emphasizing a deliberate focus on narrow scope rather than broad feature parity. The library prioritizes retrieval-augmented generation, explicitly excluding agentic tool-calling an…
-
Offline Dictation Apps Emerge as Cloud Dependence Declines
The author details how they built an offline dictation application, DictaFlow, that runs speech recognition models locally on devices like Windows laptops, Macs, and iPhones. This contrasts with services like Wispr Flow…
-
SQLite extension sqlite-vec offers dependency-free AI memory stack
A new SQLite extension called sqlite-vec is enabling developers to build AI memory stacks with significantly reduced complexity and improved performance. This extension allows vector search capabilities to be directly i…
-
StrokeSeg2 framework simplifies clinical AI deployment
Researchers have developed StrokeSeg2, a lightweight and modular C++/Qt framework designed to make deep learning-based brain lesion segmentation more accessible in clinical research. The framework adapts resource-intens…