agenta-ai/agenta
Overview
This skill audits and updates the supported LLM model list in assets.py against litellm's registry (models.litellm.ai). It ensures the playground dropdown, cost metadata, and model-to-provider mapping stay correct when adding new models or pruning outdated entries. Use it to validate model names, provider prefixes, and to run the automated guard test before committing.
How this skill works
The skill compares the canonical supported_llm_models map in sdk/agenta/sdk/assets.py to litellm.model_cost keys. It flags missing or mismatched entries, respects provider prefix conventions (e.g., anthropic/ -> claude- in litellm), and surfaces duplicates or invalid provider keys. It integrates with small Python helper scripts and a pytest guard to verify the list programmatically.
When to use it
- Adding new LLMs to the playground or cost metadata
- Removing deprecated or renamed models from the list
- Verifying model/provider prefix correctness before a release
- Auditing model coverage against litellm after a major provider update
- Running CI checks to prevent broken model dropdowns or cost lookups
Best practices
- Always run the provided check script (uvx + litellm) or the pytest guard after edits
- Fill provider-specific sets before running the missing-models audit to avoid noise
- Add new models newest-first within each provider list for clarity
- Follow provider prefix conventions (anthropic/ -> claude- lookup, cohere/ -> command- lookup)
- Cross-check Groq, DeepInfra, and Together AI lists using litellm’s provider-specific helpers
Example use cases
- A maintainer adds several OpenAI o-series and needs to confirm cost keys match litellm
- A release requires pruning old dated snapshots and verifying no dropdown regressions
- Detecting a mismatched provider enum value that would break Secrets API lookups
- Auditing Gemma/Gemini model additions and placing Gemini 1.5 models under the gemini key
- Ensuring Groq model names match the rotating groq catalogue before deployment
FAQ
The audit accepts either the direct litellm key or the provider-prefixed form; anthropic and cohere are special-cased where the prefix is stripped for cost lookup.
How do I handle frequently rotating catalogs like Groq?
Cross-check litellm.groq_models and prefer their current canonical names; re-run the check script after any Groq updates.
Which tests must pass before committing?
Run ruff format/check, then run the model validation script or the pytest guard; all checks must pass to avoid runtime dropdown or cost lookup failures.
4 skills
This skill audits and updates the supported LLM model list in assets.py to align with litellm's registry, ensuring accuracy and zero duplicates.
This skill creates complete changelog announcements across detailed pages, summaries, and sidebar cards to publish release notes consistently.
This skill updates API reference docs by fetching the latest production OpenAPI spec and regenerating Docusaurus pages for accurate endpoints.
This skill helps you add announcement cards to the Agenta sidebar, enabling timely changelog entries and banners for features and promotions.
