Brief

A tutorial demonstrates how to construct a full observability and evaluation pipeline for LLM applications using Langfuse, an open-source platform. The guide covers tracing, prompt management, scoring, and experiment execution, offering a practical workflow. It supports integration with OpenAI or a deterministic mock LLM, allowing users to explore Langfuse features without requiring paid model access. AI

STRIDE-GPT is an open-source tool designed to generate STRIDE threat models for AI applications by analyzing architecture descriptions. It emphasizes treating LLM-specific assets like system prompts, RAG documents, and agent reasoning chains as first-class components in the threat modeling process. The tool requires detailed architecture descriptions, including components, data flows, and trust boundaries, to produce effective security models. Additionally, it highlights the importance of comprehensive logging for post-incident reconstruction and suggests layered rate limiting strategies to prevent token drain attacks. AI

Pingoni has launched a new feature for its API monitoring service that tracks costs associated with OpenAI's LLM usage. This tool allows developers, particularly solo developers and small teams, to monitor their OpenAI API spend per user and per feature in real-time. The integration is designed to be simple, requiring only a few minutes to set up alongside existing API monitoring. AI