ONNX
PulseAugur coverage of ONNX — every cluster mentioning ONNX across labs, papers, and developer communities, ranked by signal.
14 day(s) with sentiment data
-
Liquid AI releases on-device agentic model LFM2.5-2.6B with 128K context
Liquid AI has released LFM2.5-2.6B, an open-weights, on-device agentic model designed for mobile and edge devices. This model boasts 2.69 billion parameters, a 128,000-token context window, and can perform multi-step ta…
-
Opt.Gear model offers 4.9x faster inference for on-device AI
Researchers have introduced Opt.Gear, a new foundation model optimized for on-device deployment and real-time inference. The model features a hybrid architecture combining convolutional and local-global attention mechan…
-
On-device AI bottleneck shifts from inference to input processing
Developers optimizing on-device AI models often focus on inference time, but one developer discovered this was not the primary bottleneck. The real performance issue lay in input processing, particularly on lower-end de…
-
DT Engineering rebuilds real-time bidding pipeline with ONNX
This two-part series details how DT Engineering rebuilt its real-time bidding pipeline using ONNX. The articles explain the rationale behind adopting ONNX, focusing on its ability to streamline model deployment and impr…
-
Kmemo 2.0 semantic cache closes performance gaps, rivals GPTCache
Kmemo 2.0, a semantic cache for LLM calls, has been released, addressing two previously identified gaps. The first gap, concerning the effectiveness of its verifier model, has been quantified: it stops a significant por…
-
Canonical Knowledge Structure (CKS) aims to bring Git-like version control to LLM knowledge
A new knowledge management system called Canonical Knowledge Structure (CKS) has been developed to address the infrastructure problems associated with Large Language Models (LLMs), such as unreliable facts and disappear…
-
ML inference on edge devices simplified with Vulkan backend
A software engineer from PostSlate detailed their experience implementing vendor-agnostic machine learning inference on production edge devices. To achieve broad compatibility across various hardware like NVIDIA, AMD, a…
-
New dataset and models enhance AI navigation for visually impaired pedestrians
Researchers have developed a new framework for semantic segmentation aimed at improving assistive navigation for visually impaired pedestrians. This framework utilizes a novel dataset called SENSATION-DS, featuring ches…
-
Visionary app streamlines AI dataset creation on macOS
Visionary is a new, locally-run macOS application designed to streamline the process of building and curating training datasets for AI models. It consolidates functionalities from multiple existing tools, offering featu…
-
Cohere enhances Arabic AI, expands model support, and highlights community contributions
Cohere has announced several updates and partnerships aimed at improving AI capabilities, particularly for under-resourced languages. The company is collaborating with HUMAIN to advance Arabic AI and has released a new …
-
Production RAG Systems: A Practical Guide to Data Indexing and Retrieval
This guide details the engineering challenges and best practices for deploying Retrieval-Augmented Generation (RAG) systems in production. It covers critical aspects such as data indexing with advanced chunking strategi…
-
EEG seizure detection models made efficient with quantization and pruning
Researchers have developed methods to make deep neural networks more efficient for detecting seizures from EEG data. They explored converting a CNN into a spiking neural network, pruning EEG channels, and using INT8 qua…
-
PrismShine tool released to verify LLM agent answers against evidence
A new tool called PrismShine has been released to address the limitations of existing hallucination checkers in LLM agents. Unlike tools that only evaluate the final output, PrismShine analyzes runtime evidence to ident…
-
Open-source tools combat LLM prompt injection attacks
Two open-source projects have been released to combat prompt injection attacks against large language models. The first, developed by BordairAPI, features a two-stage detection system combining a fast regex gate with a …
-
New framework enables private, low-latency LLM inference across edge and cloud
Researchers have developed a novel framework for edge-cloud collaborative inference of large language models (LLMs) that addresses the trade-offs between latency, resource limitations, and user privacy. This system util…
-
Automated pipeline streamlines remote-to-edge AI benchmarking
This article details a method for streamlining the AI development process, specifically for edge devices. It addresses the common challenge of a three-tier network bottleneck that separates the AI server, developer work…
-
New benchmark system for live Sikh Kirtan captioning released
Researchers have developed a new benchmark and reference system for live captioning of Sikh Kirtan, a sung recitation of verses from the Guru Granth Sahib. This task is unique as it requires exact, word-for-word accurac…
-
Sverklo offers verifiable code context for AI agents
Sverklo is a local-first MCP server designed to provide agents with verifiable code context, addressing a common failure point where agents lack traceable information about their code facts. It combines repository index…
-
RAG evaluation metrics: Hit Rate and MRR improve search quality
This article details a method for evaluating Retrieval-Augmented Generation (RAG) systems using Hit Rate and Mean Reciprocal Rank (MRR) metrics. The author created a benchmark with 360 questions and 72 lesson pages, tes…
-
Voice assistant uses CPU for ASR/TTS, freeing GPU for LLM
A user has successfully implemented a voice assistant that runs speech recognition and text-to-speech models on a CPU, freeing up the GPU for the primary LLM. They tested the Qwen3-ASR and Kokoro-TTS ONNX models on a 20…