letta-ai/letta

Overview

This skill is a compact reference guide for how major LLM providers report token usage and how prefix caching affects counts. It focuses on OpenAI, Anthropic, and Gemini differences so you can debug token accounting and optimize caching. Use it when you need accurate total token metrics or to tune prefix caching behavior across providers.

How this skill works

The guide summarizes each provider's token-count semantics, which usage fields include cached tokens, and which cache fields you must add to compute totals. It also documents minimum token sizes, how prefix caching is enabled, TTLs, and fields for reasoning or thinking tokens. The content is organized for quick lookup and applied when reconciling billing, debugging unexpected token totals, or implementing cross-provider telemetry.

When to use it

• Debugging mismatched token totals between your app and provider usage reports.
• Calculating exact total_input when using Anthropic (which excludes cached tokens).
• Designing or tuning prefix caching to reduce input token consumption.
• Comparing reasoning/thinking token fields across providers for observability.
• Implementing cross-provider billing or cost forecasting for agents.

Best practices

• Always check provider-specific fields: OpenAI and Gemini include cached tokens in input totals, Anthropic does not.
• For Anthropic, compute total_input = input_tokens + cache_read_input_tokens + cache_creation_input_tokens.
• Track reasoning/thinking fields separately (OpenAI: reasoning_tokens, Gemini: thoughts_token_count) for model diagnostics.
• Enable and validate prefix caching behavior in staging to observe TTL and minimum token effects before production.
• Standardize your telemetry schema to include raw provider fields and computed total_input to avoid ambiguity.

Example use cases

• A developer sees lower-than-expected Anthropic input_tokens and adds cache_read/cache_creation tokens to reconcile costs.
• An ops engineer tunes prefix cache TTL after noticing OpenAI cached_tokens reduced repeated prompt costs.
• A monitoring dashboard shows reasoning token usage per request by reading reasoning_tokens (OpenAI) or thoughts_token_count (Gemini).
• A cost forecaster normalizes token metrics across providers before projecting monthly spend for stateful agents.