Getting Startedintermediate
Apache Airflow 3.1.8 — Latest Changes
Explore the latest updates, features, and improvements in Apache Airflow 3.1.8, including crucial scheduler fixes, Task SDK advantages, and security patches.
Prashant Singh4 min read
All Articles
In-depth tutorials, DAG patterns, best practices, and news for Apache Airflow and the modern data engineering ecosystem.
All🌊Airflow Basics🔷DAG Patterns☸️Airflow + Kubernetes🔄Airflow + dbt📊Monitoring & Alerts🔒Security⚡Performance✅Data Quality📰News & Releases
6 articles
Sorted by newest
🌊
Airflow Basicsbeginner
Writing Your First Apache Airflow DAG
A hands-on beginner's guide to creating your first DAG in Apache Airflow. We cover the core concepts — tasks, operators, scheduling, and dependencies — with complete, runnable code.
Prashant Singh5 min read
🔷
DAG Patternsintermediate
Best Practices for Task Retries in Apache Airflow
Learn how to configure robust retry strategies in Airflow using retries, retry_delay, exponential backoff, and on_failure_callback. Includes real-world patterns for handling flaky APIs, database timeouts, and more.
Neha Agarwal5 min read
☸️
Airflow + Kubernetesadvanced
Running Apache Airflow on Kubernetes with the KubernetesExecutor
A complete production guide to deploying Apache Airflow on Kubernetes. Covers the official Helm chart, KubernetesExecutor vs CeleryExecutor, resource quotas, pod templates, and GitSync DAG deployment.
Prashant Singh6 min read
🔄
Airflow + dbtintermediate
Orchestrating dbt with Apache Airflow: A Complete Integration Guide
Learn how to integrate Apache Airflow with dbt (data build tool) using the dbt Cloud provider and Cosmos. Covers scheduling dbt runs, handling failures, passing artifacts between runs, and best practices for the dbt + Airflow stack.
Neha Agarwal6 min read
🔷
DAG Patternsintermediate
Dynamic Task Mapping in Airflow 2.x: Scale Tasks at Runtime
Master Airflow's Dynamic Task Mapping feature to create a variable number of task instances at runtime. Covers .expand(), .partial(), expand_kwargs(), and real-world patterns for parallel data processing.
Prashant Singh5 min read