astronomer/agents
Overview
This skill helps deploy Airflow DAGs and complete Airflow projects to staging and production. It covers managed deployments with Astro and self-hosted deployments using Docker Compose for development or Kubernetes with the official Helm chart for production. The guidance focuses on practical commands, CI/CD patterns, and deployment strategies to minimize downtime and speed iteration.
How this skill works
The skill inspects your deployment context (Astro vs open-source) and recommends appropriate commands and pipelines. For Astro it shows CLI deployment modes (full, DAG-only, image-only, dbt) and branch-to-environment mapping. For self-hosted setups it outlines Docker Compose quick starts for local/dev and Helm values and git-sync patterns for Kubernetes production deployments. It also highlights common operational commands and upgrade steps.
When to use it
- You need to push DAG changes quickly without rebuilding images (fast iteration).
- You want a managed CI/CD path with branch-to-environment mapping (Astro).
- You're setting up a local development environment with Docker Compose.
- You need a production-grade Airflow on Kubernetes using the Helm chart.
- You want to design CI jobs that separate image builds from DAG pushes.
Best practices
- Use astro deploy --dags for DAG-only changes and astro deploy for dependency or image changes.
- Map branches to environments (main->prod, develop->staging) and automate deploys from CI.
- For Kubernetes production, prefer git-sync or PV mounts over baking DAGs into images.
- Pin Airflow image versions for reproducible releases and use Helm values.yaml for config.
- Keep requirements changes in CI image builds and avoid ad-hoc pip installs on running clusters.
Example use cases
- Fast feature iteration: run astro deploy --dags on a feature branch to validate DAG logic quickly.
- Production release: CI merges to main trigger astro deploy (full) or Helm upgrade with a pinned image tag.
- Local testing: bring up an Airflow 3 stack with docker compose up -d for development and debugging.
- Kubernetes deploy: install or upgrade Airflow via helm install/upgrade and use git-sync for DAG distribution.
- Multi-repo CI: build and push an image from one pipeline (astro deploy --image) and push DAGs from another (astro deploy --dags).
FAQ
Rebuild when you change dependencies, requirements.txt, plugins, the Dockerfile, or any non-DAG code that must be inside the image.
Which DAG deployment method is recommended for production on Kubernetes?
Use git-sync for automatic, reliable DAG sync from a repository. Persistent volumes or baked images are alternatives but git-sync is typically easiest to manage.
15 skills
This skill helps you manage Airflow workflows by listing, testing, running, debugging DAGs, and inspecting logs, connections, and health.
This skill guides you to annotate Airflow tasks with inlets and outlets to capture data lineage for datasets and assets.
This skill helps convert a dbt Core project into an Airflow DAG or TaskGroup using Cosmos, with setup validation and execution configuration.
This skill helps you deploy dbt Fusion projects with Cosmos by validating version, installing the Fusion binary, and configuring local execution.
This skill enables human-in-the-loop decision gates in Airflow by managing approvals, forms, and branching through HITL operators for deferrable workflows.
This skill helps you deploy Airflow DAGs and projects to production using Astro or Docker/Kubernetes CI/CD, streamlining deployment strategies.
This skill generates a comprehensive table profile, including metadata, statistics, quality assessment, and practical queries to understand structure, content,
This skill helps you test, debug, and fix Airflow DAGs iteratively, guiding you through trigger, diagnosis, and retest cycles.
This skill initializes and configures Astro Airflow projects with dependencies, connections, and project structure, accelerating setup and deployment.
This skill analyzes downstream data impacts and identifies blast radius before changes, detailing affected tables, DAGs, dashboards, and owners.
This skill helps you manage your local Airflow environment with the Astro CLI, enabling start, stop, logs, and troubleshooting commands.
This skill guides you in authoring and validating Airflow DAGs using af CLI, applying best practices and reliable debugging workflows.
This skill performs comprehensive DAG failure diagnosis and root cause analysis for Airflow pipelines, delivering actionable remediation and prevention
This skill traces upstream data lineage by identifying sources, DAGs, and transformations to reveal data origins and feeding paths.
This skill initializes warehouse schema discovery, generating a ready-to-edit .astro/warehouse.md with table metadata for instant lookups and drift-aware
