https://github.com/data-engineering-community/data-engineering-project-template

This is a template you can use for your next data engineering portfolio project.
https://github.com/data-engineering-community/data-engineering-project-template

data data-engineering data-warehouse etl etl-pipeline python sql

Last synced: about 2 months ago
JSON representation

This is a template you can use for your next data engineering portfolio project.

• Host: GitHub
• URL: https://github.com/data-engineering-community/data-engineering-project-template
• Owner: data-engineering-community
• License: mit
• Created: 2021-09-10T05:13:04.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2021-09-10T05:56:53.000Z (over 3 years ago)
• Last Synced: 2024-05-01T11:29:16.205Z (8 months ago)
• Topics: data, data-engineering, data-warehouse, etl, etl-pipeline, python, sql
• Homepage: https://dataengineering.wiki
• Size: 137 KB
• Stars: 117
• Watchers: 3
• Forks: 28
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Data Engineering Project Template

This is a template you can use for your next data engineering portfolio project. To copy it, log into GitHub and click on the **Use this template** button above.

![GitHub use this template button](use-this-template-button.png)

## Overview

Here you want to write a short overview of the goals of your project and how it works at a high level. If possible, include one or two images of the end product and architecture diagram (see examples below). diagrams.net is a great tool for creating architecture diagrams.

### Data Visualization

![Example dashboard image](example-dashboard.png)

### Data Architecture

![Example architecture image](example-architecture.png)

If you decide to include this, you should also talk a bit about why you chose the architecture and tools you did for this project.

## Prerequisites

Directions or anything needed before running the project.

- Prerequisite 1

- Prerequisite 2

- Prerequisite 3

## How to Run This Project

Replace the example step-by-step instructions with your own.

1. Install x packages

2. Run command: `python x`

3. Make sure it's running properly by checking z

4. To clean up at the end, run script: `python cleanup.py`

## Lessons Learned

It's good to reflect on what you learned throughout the process of building this project. Here you might discuss what you would have done differently if you had more time/money/data. Did you end up choosing the right tools or would you try something else next time?

## Contact

Please feel free to contact me if you have any questions at: LinkedIn, Twitter