Span attributes that catch LLM cost regressions before billing does
A developer has detailed how to implement custom span attributes within OpenTelemetry to track Large Language Model (LLM) costs more effectively. By adding attributes like `team.id`, `feature.id`, and `llm.model` to spans, organizations can identify cost regressions at a granular level, such as per team or per feature, before they significantly impact billing. This approach, utilizing tools like Tempo and Grafana for querying and alerting, proved more effective than standard trace-level debugging or org-level dashboards in catching unexpected cost spikes. AI
IMPACT Provides a practical method for developers to monitor and control LLM operational expenses, crucial for efficient AI deployment.