authoring-dags_skill

Overview

This skill guides writing and validating Apache Airflow DAGs with a practical, CLI-driven workflow. It covers discovery, planning, implementing, validating, and iterating on DAGs, and points to the testing-dags skill for full test/debug cycles. Use it to produce consistent, maintainable pipeline code that fits the target Airflow environment.

How this skill works

It inspects the codebase and Airflow runtime using af CLI commands (run via uvx) to discover installed providers, connections, variables, and existing DAG patterns. After planning a DAG structure and dependencies, you implement the DAG file, update requirements if needed, then validate with parse error checks, DAG inspection, and warnings. Validation and iterative fixes are performed using the af commands as a feedback loop before testing.

When to use it

• Creating a new DAG and aligning it with repository conventions
• Choosing operators and connections based on installed providers
• Validating parse errors, DAG metadata, and task structure after saving a DAG file
• Preparing a DAG for a formal test run (use testing-dags for execution details)
• Proposing DAG structure and resource needs for review or approval

Best practices

• Discover first: inspect **/dags/** patterns, requirements.txt, and runtime config via af config commands
• Plan tasks, dependencies, schedule, required connections, and packages before coding
• Keep DAG files small and idempotent; avoid runtime side effects at import time
• Use af dags errors, af dags get, af dags warnings, and af dags explore as a validation loop
• Ask for user consent before triggering runs and use the testing-dags skill for full test/debug workflows

Example use cases

• Add a new daily ETL DAG that uses an existing Snowflake connection found via af config connections
• Refactor DAGs to use provider operators confirmed by af config providers to avoid custom hooks
• Validate that a newly committed DAG parses cleanly and has correct schedule and tags using af dags get and af dags explore
• Plan and document required variable and connection changes before implementation for a team review
• Quickly iterate on fixes after af dags errors shows import or dependency issues