.@DecagonAI cut voice agent cost per turn nearly 6x with Together AI.
DecagonAI has significantly reduced the cost of its voice agent by nearly sixfold by migrating from closed models to fine-tuned open-source models hosted on Together AI. This transition maintained low latency for real-time voice interactions, achieving under 400ms p95 model latency per turn. The optimization involved custom speculators, prompt caching, and deployment on NVIDIA Blackwell hardware, enabling frequent model updates. AI
IMPACT Demonstrates significant cost efficiencies and performance gains achievable by migrating from proprietary to fine-tuned open-source models for specialized applications.