langsmith-fetch_skill

Overview

This skill helps debug LangChain and LangGraph agents by fetching execution traces from LangSmith Studio into your terminal. It automates retrieval of recent traces, inspects tool calls, memory operations, errors, token usage, and execution timing. Requires the langsmith-fetch CLI and proper LANGSMITH environment variables to work.

How this skill works

The skill runs langsmith-fetch commands to collect traces, individual trace JSON, or thread exports, then analyzes the results for errors, tool call sequences, memory activity, token counts, and latencies. It offers quick summaries (pretty format) for fast inspection and JSON/raw outputs for deep analysis or automation. When issues are detected it provides root-cause pointers and suggested fixes based on trace events and tool failures.

When to use it

• You need to debug unexpected agent behavior or failures
• Investigating errors, exceptions, or timeouts reported by an agent
• Verifying memory operations: stores, recalls, and their usage
• Analyzing agent performance, token usage, or slow tool calls
• Exporting a debug session to share with the team or archive traces

Best practices

• Install and verify langsmith-fetch before starting (pip install langsmith-fetch)
• Set LANGSMITH_API_KEY and LANGSMITH_PROJECT in your environment and confirm they persist
• Use --format pretty for quick human review and --format json for automated parsing
• Export sessions into timestamped folders and save failing traces under an error-cases directory
• Run short health checks after code or config changes (last-n-minutes 5)

Example use cases

• Quick debug: run langsmith-fetch traces --last-n-minutes 5 --limit 5 --format pretty and report failures, tools called, duration, and tokens
• Deep dive: fetch a trace JSON with langsmith-fetch trace <id> --format json and identify the failing tool, error message, and suggested fix
• Export session: create a session folder and export traces and threads for sharing or archival
• Error analysis: collect recent traces in JSON and grep for error/exception to summarize error types, frequency, and affected agents
• Memory diagnosis: search traces for memory store/recall operations to confirm whether memories were persisted and used