phoenix-tracing_skill

Overview

This skill provides OpenInference semantic conventions and instrumentation guidance for Phoenix AI observability. It helps engineers implement LLM tracing, create custom spans, and deploy tracing safely to production. The content targets Python and TypeScript integrations and includes span schemas, setup steps, and production considerations.

How this skill works

The skill defines rule-based files that describe setup, auto and manual instrumentation, and nine standardized span types with required attributes. It inspects LLM operations, retrievals, agents, tools, and evaluation steps and maps them to OpenInference attributes for consistent tracing. It also provides organization, enrichment, and production guidance (batching, masking) to make traces queryable and safe.

When to use it

• Setting up Phoenix tracing for a new LLM application (Python or TypeScript)
• Enabling auto-instrumentation for frameworks like LangChain, LlamaIndex, or custom clients
• Creating manual/custom spans for pipelines, agents, or tool calls
• Adding OpenInference attributes for consistent analytics and cost tracking
• Preparing tracing and masking rules before deploying to production

Best practices

• Start with the setup-{lang} rules to install and configure OTEL and the Phoenix exporter
• Use auto-instrumentation where available, then add manual spans for business-critical steps
• Follow span-{type} attribute schemas (LLM, RETRIEVER, TOOL, etc.) to ensure interoperability
• Include universal attributes (user.id, session.id, llm.model_name, token counts) for cross-trace queries
• Apply production rules for batching, PII masking, and error handling before deployment

Example use cases

• Trace an end-to-end chat pipeline: client input → retriever → reranker → LLM response using span-chain and span-llm
• Instrument a reasoning agent to create span-agent entries for each reasoning step and tool.invocations
• Add custom metadata to group traces by project or session for product-level analytics
• Deploy tracing in production with batching and PII masking to maintain performance and compliance
• Use evaluator spans to record automated LLM evaluation metrics and human annotations