---
name: airflow-dag-patterns
description: "Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs."
risk: safe
source: community
date_added: "2026-02-27"
---

# Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

## Use this skill when

- Creating data pipeline orchestration with Airflow
- Designing DAG structures and dependencies
- Implementing custom operators and sensors
- Testing Airflow DAGs locally
- Setting up Airflow in production
- Debugging failed DAG runs

## Do not use this skill when

- You only need a simple cron job or shell script
- Airflow is not part of the tooling stack
- The task is unrelated to workflow orchestration

## Instructions

1. Identify data sources, schedules, and dependencies.
2. Design idempotent tasks with clear ownership and retries.
3. Implement DAGs with observability and alerting hooks.
4. Validate in staging and document operational runbooks.

Refer to `resources/implementation-playbook.md` for detailed patterns, checklists, and templates.

## Safety

- Avoid changing production DAG schedules without approval.
- Test backfills and retries carefully to prevent data duplication.

## Resources

- `resources/implementation-playbook.md` for detailed patterns, checklists, and templates.

## Limitations
- Use this skill only when the task clearly matches the scope described above.
- Do not treat the output as a substitute for environment-specific validation, testing, or expert review.
- Stop and ask for clarification if required inputs, permissions, safety boundaries, or success criteria are missing.


---
Source attribution: wshobson
Original project: https://github.com/wshobson/agents
License: CC-BY-4.0
License information: https://creativecommons.org/licenses/by/4.0/
Original upstream instructions; downloaded from Dave AI.


# Dave AI resource metadata

Creator / publisher: wshobson
Original project: https://github.com/wshobson/agents
Attribution basis: Historical import commit and explicit upstream port attribution

Fields of work: Development & Engineering
Roles: Data Engineer
Specialties: Specialty not specified
Tools: Tool not specified
Country applicability: Country not specified
Classification basis: Dave AI task classification from package title, description, and declared metadata
Instruction SHA-256: 94a41e7557ed0038de304d942691a6f432f6a29b82dc096e7c1f265b74fc332c
Metadata reviewed: 2026-10-27
This metadata supersedes earlier Dave AI catalog attribution. Original instruction and supporting files are preserved.