langsmith
PulseAugur coverage of langsmith — every cluster mentioning langsmith across labs, papers, and developer communities, ranked by signal.
- acquired by Promptfoo 95%
- developed by langchain-openai 90%
- competes with Langfuse 70%
- competes with Arize Phoenix 70%
- competes with DeepEval 70%
- used by Promptfoo 70%
- affiliated with langchain-core 70%
- used by vcrpy 70%
- competes with Phoenix 70%
- affiliated with langchain-openai 70%
- competes with Future AGI 70%
- used by aiohttp 70%
- 2026-08-25 product_launch Fireworks AI and LangChain are co-hosting a workshop on August 25th focused on building custom evaluation models for LangSmith. source
- 2026-08-25 product_launch Fireworks and LangChain are co-hosting a workshop to demonstrate building custom evaluation models for LangSmith. source
- 2026-05-28 product_launch AWS and LangChain released a guide detailing how to use LangSmith on AWS for evaluating AI agents. source
11 day(s) with sentiment data
-
LLM observability tools capture data but fail to judge agent output quality
The article discusses the evolution of LLM infrastructure, moving from direct vendor SDKs to LLM gateways and dedicated observability stacks. While gateways like LiteLLM and Portkey simplify model switching, observabili…
-
Top 5 LLM Evaluation Frameworks for Release Engineering Ranked
A recent analysis highlights Promptfoo as the leading LLM evaluation framework for release engineering, particularly for its CI/CD integration that can block builds on failed tests. DeepEval is recommended for Python-ba…
-
Fireworks and LangChain Host Workshop on LangSmith Evaluation Models
Fireworks and LangChain are co-hosting a workshop on August 25th focused on building custom evaluation models for LangSmith. The event aims to help attendees scale their observability stacks to enhance performance. A ro…
-
ZizkaDB launches to debug LLM agent decision chains
ZizkaDB has launched as an open-source operational database designed to address the debugging challenges of LLM agents. Unlike traditional tracing tools that provide a span tree of events, ZizkaDB stores agent decisions…
-
LangChain updates Anthropic integration with new features and fixes
LangChain has released two new versions of its Anthropic integration: 1.6.1 and 1.6.0. Version 1.6.1 includes a fix for filtering invalid tool calls from Anthropic's v1 content. Version 1.6.0, released prior to 1.6.1, i…
-
LangSmith adds Tuned Evaluators to improve AI agent quality feedback
LangSmith has introduced Tuned Evaluators, a feature designed to enhance the quality feedback loop for AI agents by attaching specific quality metrics to production traces. Initially, this feature focuses on 'Perceived …
-
LLM observability tools diverge, focusing on distinct core problems
The LLM observability landscape is diversifying, with tools like LangSmith, Langfuse, Braintrust Ai, and Helicone each focusing on different core problems rather than competing directly. LangSmith emphasizes tracing Lan…
-
LangChain updates OpenAI integration to v1.5.2 with new features and fixes
LangChain has released version 1.5.2 of its langchain-openai package, introducing several fixes and features. Key updates include preserving reasoning item boundaries, extracting gateway metadata from response headers, …
-
LangChain updates OpenAI integration to v1.5.1, fixing reasoning bug
LangChain has released version 1.5.1 of its langchain-openai integration, following closely on the heels of version 1.5.0. The latest update, 1.5.1, focuses on fixing an issue where streamed encrypted reasoning was not …
-
Fireworks AI and LangChain Host Workshop on Custom Eval Models
Fireworks AI is hosting a workshop on August 25th in collaboration with LangChain. The event will focus on building custom evaluation models for LangSmith and enhancing observability stacks to improve performance. Atten…
-
LangChain releases updates to core components and main library
LangChain has released updates to its core components and the main library. Langchain-core version 1.5.5 includes fixes for pydantic validation, merging chunks, and handling Anthropic content blocks, alongside explicit …
-
LLM observability platforms diverge on advanced features as market booms
The LLM observability and evaluation platform market is rapidly expanding, with projections reaching $9.26 billion by 2030. Platforms are diversifying into AI-native tools, open-source evaluation libraries, AI gateways,…
-
LLM observability tools capture traces but limit assertion granularity
Observability tools for LLM agents, such as Langfuse, LangSmith, and Phoenix, offer ways to capture production traces, but their default configurations for defining inputs and assertions can be limiting. The author argu…
-
RAG observability tools like LangSmith and Arize Phoenix gain traction
Retrieval-Augmented Generation (RAG) systems are moving from experimental stages to critical production components for chatbots and other applications. To ensure these systems function effectively, robust observability …
-
OpenAI's Promptfoo Acquisition Sparks Debate on LLM Evaluation Independence
The acquisition of Promptfoo by OpenAI has prompted a re-evaluation of LLM evaluation tools, highlighting concerns about vendor dependency and cost. The author proposes an alternative approach using a custom-trained cla…
-
New tool converts agent failures into fine-tuning data
A new open-source tool called trace2train has been released to convert failed agent traces into supervised fine-tuning (SFT) or Direct Preference Optimization (DPO) training data. Developed as a local CLI tool, it aims …
-
LangSmith LLM Gateway adds runtime spend limits and PII redaction
LangSmith's new LLM Gateway offers runtime governance for AI agents, addressing budget and compliance risks. It integrates directly into the request path between agents and LLM providers, enabling features like hard spe…
-
LangChain Ecosystem Explained: Building Blocks vs. Operational Tools
The LLM development ecosystem is experiencing "Lang-fatigue" due to a proliferation of tools with similar naming conventions. This guide clarifies the distinctions between open-source building blocks like LangChain, Lan…
-
LLM prompt edits bypass testing, causing significant accuracy drops
A significant drop in LLM extraction accuracy, from 0.87 to 0.78, occurred after a minor one-word edit to the system prompt. This highlights a critical gap in current LLM application development, where prompt changes of…
-
OpenSmith releases major update for local LLM tracing
OpenSmith, a local-first alternative to LangSmith for tracing LLM pipelines, has released a significant update. The new version features a redesigned dashboard with real-time updates, enhanced search and filtering capab…