https://github.com/garystafford/aws-airflow-demo

Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS.
https://github.com/garystafford/aws-airflow-demo

airflow amazon-emr amazon-mwaa apache-airflow aws pyspark-applications

Last synced: 2 months ago
JSON representation

Project files for the post: Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS.

• Host: GitHub
• URL: https://github.com/garystafford/aws-airflow-demo
• Owner: garystafford
• License: apache-2.0
• Created: 2020-12-21T03:40:08.000Z (about 4 years ago)
• Default Branch: main
• Last Pushed: 2022-07-06T12:56:01.000Z (over 2 years ago)
• Last Synced: 2023-08-05T02:22:47.613Z (over 1 year ago)
• Topics: airflow, amazon-emr, amazon-mwaa, apache-airflow, aws, pyspark-applications
• Language: Python
• Homepage:
• Size: 753 KB
• Stars: 39
• Watchers: 6
• Forks: 14
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Running PySpark Applications on Amazon EMR using Apache Airflow

## Overview

Project files for the post, [Running PySpark Applications on Amazon EMR using Apache Airflow: Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS](https://itnext.io/running-spark-jobs-on-amazon-emr-with-apache-airflow-2e16647fea0c). Please see post for complete instructions on using the project's files.

## Updates

Project was updated for Amazon MWAA Airflow version 2.0.2 on 2021-11-04.

---

The contents of this repository represent my viewpoints and not of my past or current employers, including Amazon Web

Services (AWS). All third-party libraries, modules, plugins, and SDKs are the property of their respective owners. The author(s) assumes no responsibility or liability for any errors or omissions in the content of this site. The information contained in this site is provided on an "as is" basis with no guarantees of completeness, accuracy, usefulness or timeliness.