5 Best Apache Airflow Tutorials & Courses - Learn Apache Airflow Online
Are you looking for the best Apache Airflow tutorials? Here are 5 of the best Airflow tutorials to help you learn how to author, schedule and monitor data pipelines.
The Best Apache Airflow Tutorials & Courses online for beginners to Learn Apache Airflow in 2024.
Apache Airflow is a workflow automation and scheduling system used by authors to manage data pipelines. It uses workflows to create direct acyclic diagrams and DAG tasks. Workflows are defined in the Python file airflow-dAG (Direct Acyclic Graphs and More).
This tool is a workflow engine for planning and operating your complex data pipelines. It ensures that tasks in your data pipeline are executed in the correct order and that tasks receive the necessary resources. Use the airflow to organize and initiate machine learning that runs on an external Spark cluster.
Keeping this in mind, here at Coursesity, we have curated some of the Best Online Apache Airflow Tutorials & Courses with certification. Hope that you will find the best course for you to learn how to author, schedule, and monitor data pipelines.
Disclosure: Coursesity is supported by the learners community. We may earn an affiliate commission when you make a purchase via links on Coursesity.
Top Apache Airflow Tutorials & Courses List
1. The Complete Hands-On Introduction to Apache Airflow
Learn to author, schedule, and monitor data pipelines through practical examples using Apache Airflow.
In this course, you will learn how to:
- create plugins to add functionalities to Apache Airflow.
- use Docker with Airflow and different executors.
- master core functionalities such as DAGs, Operators, Tasks, Workflow, etc.
- understand and apply advanced concepts of Apache Airflow such as XCOMs, Branching, and SubDAGs.
- understand the difference between Sequential, Local, and Celery Executors, how do they work and how can you use them.
- use Apache Airflow in a Big Data ecosystem with Hive, PostgreSQL, Elasticsearch, etc.
- install and configure Apache Airflow.
- think, answer and implement solutions using Airflow to real data processing problems.
With this Apache Airflow tutorial, you will learn everything you need to start using Apache Airflow through theory and practice. Starting from very basic notions such as, what is Airflow and how it works, you will dive into advanced concepts such as, how to create plugins and make real dynamic pipelines.
You can take The Complete Hands-On Introduction to Apache Airflow certification course on Udemy.
Course rating: 4.7 out of 5.0 ( 3,786 Ratings total)
Duration: 5 h 5 m
Certificate: Certificate on completion
2. Productionalizing Data Pipelines with Apache Airflow
This course will teach you how to master production-grade Data Pipelines with ease within Apache Airflow.
The course includes:
- Introducing Apache Airflow
- Dissecting the Components of a Pipeline
- Demystifying Common DAGs Pitfalls
- Abstracting Functionality
- Scaling Airflow
In this Productionalizaing Data Pipelines with Apache Airflow tutorial, you will learn to master them using Apache Airflow. Initially, you will explore what Airflow is and how it creates Data Pipelines. Discover how to make your pipelines more resilient and predictable.
Plus, you will learn how to distribute tasks with Celery and Kubernetes Executors. When you’re finished with this course, you’ll have the skills and knowledge of Apache Airflow needed to make any Data Pipelines production-grade.
You can take Productionalizing Data Pipelines with Apache Airflow certification course on Pluralsight.
Course rating: 4.5 out of 5.0 ( 45 Ratings total)
Duration: 2 h 12 m
Certificate: Certificate on completion
3. Apache Airflow on AWS EKS: The Hands-On Guide
How to set up Apache Airflow on AWS EKS with the Kubernetes Executor and be production-ready!
In this course, you will learn how to/:
- set up a production-ready architecture for Airflow on AWS EKS from basic to advanced.
- deploy DAGs from Git (public and private).
- create CI/CD Pipelines with AWS CodePipeline Deploy DAGs.
- share DAGs and store logs with AWS EFS.
- enable remote logging with AWS S3 in EKS.
- test your DAGs in CI/CD pipelines.
- store sensitive data in AWS Secret Manager.
This Apache Airflow tutorial is designed to guide you through the different steps of creating a real-world architecture and configuring the EKS cluster following best practices. You will learn how to deploy changes automatically with GitOps.
Learn how to use Helm to configure and set up Airflow on Kubernetes and configuring the official Helm chart of Airflow to use the Kubernetes Executor and many different features.
Plus, This Apache Airflow tutorial will show you how to deploy DAGs in Airflow with Git-Sync and AWS EFS and how you can deploy DAGs/Airflow through CI/CD pipelines with AWS CodePipeline.
In addition, you will learn how to test your DAGs automatically and secure your credentials & sensitive data in a Secret Backend. Make the production environment scalable and highly available.
By the end of this tutorial, you will learn how to enable remote logging with AWS S3 and create 3 different environments dev/staging and prod
You can take Apache Airflow on AWS EKS: The Hands-On Guide certification course on Udemy.
Course rating: 4.9 out of 5.0 ( 142 Ratings total)
Duration: 9 h
Certificate: Certificate on completion
4. Apache Airflow
Learn A to Z of Apache Airflow from basic to advance level, deploy workflows & data pipelines using Airflow with Docker.
In this course, you will learn:
- the implementation of Airflow core nomenclature
- advance Airflow concepts, the explanation to which is not very clear even in Airflow's official documentation.
- exclusive features like Data Profiling, Charts, Trigger rules, AirflowIgnore file, Zombies, Undeads, and LatestOnly operator.
- creating data pipeline of Real-Time case studies using Apache Airflow.
With this Apache Airflow tutorial, you will learn the implementation of Airflow core nomenclature. i.e. DAG, Operators, Tasks, Executors, Cfg file, UI views, etc. Learn about XComs, Hooks, Pools, SubDAGs, Variables, Connections, Plugins, Adhoc queries, Sensors, etc.
Plus, you will learn about advanced Airflow concepts like Data Profiling, Charts, Trigger rules, AirflowIgnore file, Zombies, Undeads, and LatestOnly operator.
You can take the Apache Airflow certification course on Udemy.
Course rating: 4.4 out of 5.0 ( 808 Ratings total)
Duration: 6 h
Certificate: Certificate on completion
5. Apache Airflow: The Hands-On Guide
Master Apache Airflow from A to Z. Hands-on guidance on Airflow with AWS, Kubernetes, and Docker.
In this course, you will learn how to:
- code production-grade Data pipelines by mastering Apache Airflow.
- scale Airflow with the Local, Celery, and Kubernetes Executors.
- set up monitoring with Elasticsearch and Grafana.
- secure Airflow with authentication, crypto, and the RBAC UI.
- core and advanced concepts with pros and limitations.
- master DAGs with timezones, unit testing, backfill, and catchup.
- organize the DAG folder and keep things clean.
Here, you will learn the fundamentals of Airflow like what is Airflow, how the scheduler and the webserver works. Learning how to master DAGs is a top priority and you will learn to play with timezones, unit testing your DAGs, how to structure your DAG folder, and much more.
Next, you will learn how to scale Airflow through different executors such as the Local Executor, the Celery Executor, and the Kubernetes Executor. You will discover how to specialize your workers, how to add new workers, what happens when a node crashes.
Plus, you will learn advanced concepts of Airflow such as templating your DAGs, how to make your DAG dependent on another, what are Subdags and deadlocks, and more. You will set up a Kubernetes cluster in the cloud with AWS EKS and Rancher to use Airflow along with the Kubernetes Executor
You can take Apache Airflow: The Hands-On Guide certification course on Udemy.
Course rating: 4.6 out of 5.0 ( 1,384 Ratings total)
Duration: 13 h 5 m
Certificate: Certificate on completion
Hey! We hope you have found this Online Apache Airflow Tutorials & Courses with certification list helpful and intriguing. Since you've made it this far then certainly you are willing to learn more and here at Coursesity, it is our duty to enlighten people with knowledge on topics they are willing to learn.
Here are some more topics that we think will be interesting for you!