airflow-dag-patterns

Automation & Intégrations

Build production Apache Airflow DAGs with best practices for operators, sensors, testing, and deployment. Use when creating data pipelines, orchestrating workflows, or scheduling batch jobs.

Documentation

Apache Airflow DAG Patterns

Production-ready patterns for Apache Airflow including DAG design, operators, sensors, testing, and deployment strategies.

Use this skill when

●Creating data pipeline orchestration with Airflow
●Designing DAG structures and dependencies
●Implementing custom operators and sensors
●Testing Airflow DAGs locally
●Setting up Airflow in production
●Debugging failed DAG runs

Do not use this skill when

●You only need a simple cron job or shell script
●Airflow is not part of the tooling stack
●The task is unrelated to workflow orchestration

Instructions

1.Identify data sources, schedules, and dependencies.
2.Design idempotent tasks with clear ownership and retries.
3.Implement DAGs with observability and alerting hooks.
4.Validate in staging and document operational runbooks.

Refer to resources/implementation-playbook.md for detailed patterns, checklists, and templates.

Safety

●Avoid changing production DAG schedules without approval.
●Test backfills and retries carefully to prevent data duplication.

Resources

●resources/implementation-playbook.md for detailed patterns, checklists, and templates.

Compétences similaires

Explorez d'autres agents de la catégorie Automation & Intégrations

Voir tout le catalogue

blockrun

Use when user needs capabilities Claude lacks (image generation, real-time X/Twitter data) or explicitly requests external models ("blockrun", "use grok", "use gpt", "dall-e", "deepseek")

VOIR LA FICHE

microsoft-teams-automation

"Automate Microsoft Teams tasks via Rube MCP (Composio): send messages, manage channels, create meetings, handle chats, and search messages. Always search tools first for current schemas."

VOIR LA FICHE

upstash-qstash

"Upstash QStash expert for serverless message queues, scheduled jobs, and reliable HTTP-based task delivery without managing infrastructure. Use when: qstash, upstash queue, serverless cron, scheduled http, message queue serverless."

VOIR LA FICHE
Utiliser l'Agent airflow-dag-patterns - Outil & Compétence IA | Skills Catalogue | Skills Catalogue