setting-up-astro-project_skill

Overview

This skill initializes and configures Astro (Astronomer) Airflow projects to get a reproducible development environment and deployment-ready project layout. It guides creating project structure, adding Python and OS-level dependencies, and managing Airflow connections, variables, and pools. Use it to standardize setup before authoring or running DAGs locally or in CI.

How this skill works

The skill uses the Astro CLI to scaffold a project (astro dev init) that includes dags, plugins, include, tests, and configuration files. Dependencies are declared in requirements.txt and packages.txt, with optional Dockerfile customization for private indexes or build steps. Connections, variables, and pools are managed through an airflow_settings.yaml that the local environment loads automatically and can be exported/imported with astro dev object commands. Validate DAGs quickly using astro dev parse before starting the full environment.

When to use it

• Creating a new Astro/Airflow project from scratch
• Adding or pinning Python and OS-level dependencies
• Setting up Airflow connections, variables, or pools for local development
• Customizing the runtime image or adding build-time steps via Dockerfile
• Exporting/importing connection and variable configs between environments

Best practices

• Keep requirements.txt minimal and pin versions to ensure reproducible builds
• List only needed OS packages in packages.txt to reduce image size
• Store connections and variables in airflow_settings.yaml for repeatable setup and version control (avoid storing secrets in plaintext)
• Use a custom Dockerfile only when necessary; prefer pip extra-index for private packages
• Run astro dev parse to catch DAG syntax or import errors before starting the environment

Example use cases

• Onboard a new data-engineering repo by running astro dev init and committing the standard project layout
• Add a Snowflake provider and pandas to requirements.txt, then astro dev restart to apply changes
• Set a Postgres connection and an env variable in airflow_settings.yaml to mirror production settings for local testing
• Export connections from CI or a teammate with astro dev object export and import them into your local environment
• Create a Dockerfile to install private packages from an internal PyPI and build a custom runtime image