https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint

Companion repository for the "Streamlining AWS Glue CI/CD — A Comprehensive Blueprint" blog post
https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint

aws aws-glue ci-cd dataops devops iac-terraform infrastructure-as-code terraform

Last synced: 2 months ago
JSON representation

Companion repository for the "Streamlining AWS Glue CI/CD — A Comprehensive Blueprint" blog post

• Host: GitHub
• URL: https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint
• Owner: ricardolsmendes
• License: mit
• Created: 2023-08-14T14:29:22.000Z (almost 2 years ago)
• Default Branch: dev
• Last Pushed: 2024-11-08T18:23:19.000Z (8 months ago)
• Last Synced: 2025-04-24T06:06:37.952Z (2 months ago)
• Topics: aws, aws-glue, ci-cd, dataops, devops, iac-terraform, infrastructure-as-code, terraform
• Language: HCL
• Homepage: https://blog.det.life/streamlining-aws-glue-ci-cd-298edda9e844
• Size: 518 KB
• Stars: 12
• Watchers: 1
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Contributing: .github/CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: .github/CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
# AWS Glue CI/CD Blueprint

This is the companion repository for the [Streamlining AWS Glue CI/CD — A Comprehensive

Blueprint](https://blog.det.life/streamlining-aws-glue-ci-cd-298edda9e844) blog post. It

contains a reference CI/CD pipeline for building, testing, and deploying data processing

jobs in [AWS Glue](https://aws.amazon.com/glue/).

[![license](https://img.shields.io/github/license/ricardolsmendes/aws-glue-ci-cd-blueprint.svg)](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/LICENSE)

[![issues](https://img.shields.io/github/issues/ricardolsmendes/aws-glue-ci-cd-blueprint.svg)](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/issues)

[![continuous integration](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/actions/workflows/on-push-to-main.yaml/badge.svg)](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/actions/workflows/on-push-to-main.yaml)

Coverage Report FileStmtsMissCoverMissingsrc   s3_json_to_parquet_job.py21195%72   s3_to_s3_job.py21195%60TOTAL42295% 

## 1. Overview

![N|Solid](img/glue-ci-cd-blueprint-overview.png "Overview of the AWS Glue CI/CD Blueprint")

### 1.1. Continuous Integration / Continuous Delivery

- [GitHub Actions](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/.github/workflows)

### 1.2. Infrastructure as Code

- [Terraform](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/infrastructure)

### 1.3. Quality Assurance

- [Python unit tests: unittest + pytest](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/tests)

- Python formatter: [Black](https://github.com/psf/black)

- Python linter: [Flake8](https://github.com/PyCQA/flake8)

## 2. How to contribute

Please make sure to take a moment and read the [Code of

Conduct](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/.github/CODE_OF_CONDUCT.md).

### 2.1. Report issues

Please report bugs and suggest features via the [GitHub

Issues](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/issues).

Before opening an issue, search the tracker for possible duplicates. If you find a

duplicate, please add a comment saying that you encountered the problem as well.

### 2.2. Contribute code

Please make sure to read the [Contributing

Guide](https://github.com/ricardolsmendes/aws-glue-ci-cd-blueprint/blob/main/.github/CONTRIBUTING.md)

before making a pull request.