Data Engineering Project SQL, Python, Airflow, Docker, CI/CD
Size: 3.00 GB

Data Engineering is the backbone of modern data-driven companies. To excel, you need experience with the tools and processes that power data pipelines in real-world environments. This course gives you practical, project-based learning with the following tools PostgreSQL, Python, Docker, Airflow, Postman, SODA and Github Actions. I will guide you as to how you can use these tools.

What you will learn in the course:

  1. Python for Data Engineering: Build Python scripts for data extraction by interacting with APIs using Postman, loading into the data warehouse and transforming (ELT). In this course we use Python version 3.10.
  2. SQL for Data Pipelines: Use PostgreSQL as a data warehouse. Interact with the data warehouse using both psql & DBeaver
  3. Docker for Containerized Deployments: Discover how to containerize data applications using Docker, making your data pipelines portable and easy to scale.
  4. Airflow for Workflow Automation: Master the basics of orchestrating and automating your data workflows with Apache Airflow, a must-have tool in data engineering. In this course we use Airflow version 2.9.2.
  5. Testing and Data Quality Assurance: Understand how to perform unit, integration & end-to-end (E2E) tests using a combination of pytest and Airflow’s DAG tests to validate your data pipelines. Implement data quality tests using SODA to ensure your data meets business and technical requirements.
  6. CI/CD for Automated Testing & Deployment: Learn to automate deployment pipelines using GitHub Actions to ensure smooth, continuous integration and delivery.

HOMEPAGE – https://www.udemy.com/course/start-your-data-engineering-journey-project-based-learning/

Free Download Link-

Note: Comment below if you find the download link dead.


0 Comments

Leave a Reply

Avatar placeholder

Your email address will not be published. Required fields are marked *