https://github.com/airscholar/cicd_for_data_engineering

This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study
https://github.com/airscholar/cicd_for_data_engineering

azure infrastructure-as-code terraform

Last synced: 2 months ago
JSON representation

This project showcases how to integrate the world of DevOps, focusing on Continuous Integration (CI) and Continuous Deployment (CD) with the realm of modern data engineering using Terraform and Azure as the case study

• Host: GitHub
• URL: https://github.com/airscholar/cicd_for_data_engineering
• Owner: airscholar
• Created: 2023-12-27T08:30:46.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-12-27T09:04:22.000Z (about 1 year ago)
• Last Synced: 2024-04-18T02:57:11.023Z (9 months ago)
• Topics: azure, infrastructure-as-code, terraform
• Language: HCL
• Homepage: https://youtu.be/bCwLJUAqmC0
• Size: 55.7 KB
• Stars: 7
• Watchers: 3
• Forks: 3
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
This repository is an Infrastructure as Code (IaC) project that leverages terraform for orchestrating the provisioning of the Azure services related to Data Engineering.

## Prerequisites

1. Azure Account with a subscription

2. Azure CLI

3. Terraform CLI

4. Any IDE of your choice (VS Code, PyCharm, etc.)

## System Architecture

![system_architecture.jpg](images%2Fsystem_architecture.jpg)

## Getting Started

1. Clone this repository

2. Replace the content of the file named `terraform.tfvars` in the root directory of the project

3. Run the following commands in the root directory of the project:

    - `terraform init` (to initialize the terraform project)

    - `terraform validate` (to validate the terraform code)

    - `terraform plan` (to see the changes that will be applied)

    - `terraform apply` (to apply the changes)

    - `terraform destroy` (to destroy the infrastructure)

## Video

[![CICD for Modern Data Engineering](https://img.youtube.com/vi/bCwLJUAqmC0/0.jpg)](https://youtu.be/bCwLJUAqmC0)