https://github.com/wittline/data-engineer-challenge
Challenge Data Engineer
https://github.com/wittline/data-engineer-challenge
data-engineering data-pipeline dataengineering docker docker-compose fastapi postgresql
Last synced: 3 months ago
JSON representation
Challenge Data Engineer
- Host: GitHub
- URL: https://github.com/wittline/data-engineer-challenge
- Owner: Wittline
- License: apache-2.0
- Created: 2022-06-03T04:29:49.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2022-06-13T15:21:30.000Z (about 3 years ago)
- Last Synced: 2025-04-13T06:09:35.813Z (3 months ago)
- Topics: data-engineering, data-pipeline, dataengineering, docker, docker-compose, fastapi, postgresql
- Language: Python
- Homepage: https://wittline.github.io/data-engineer-challenge/
- Size: 31.3 KB
- Stars: 25
- Watchers: 2
- Forks: 8
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Data Engineer Challenge
This goal of this repository is based on solving a technical challenge for the data engineering position.
Check the article here: Design, Development and Deployment of a simple Data Pipeline
- Install Docker Desktop on Windows, it will install Docker Compose as well, Docker Compose will allow you to run multiple container applications.
- Install git-bash for windows, once installed, open git bash and download this repository, this will download the docker-compose.yaml file, and other files needed.
```linux
ramse@DESKTOP-K6K6E5A MINGW64 /c
$ git clone https://github.com/Wittline/data-engineer-challenge.git
```
- Once all the files needed were downloaded from the repository, let's run everything. We will use the git bash tool again, go to the folder data-engineer-challenge and we will run the Docker Compose command:
```linux
ramse@DESKTOP-K6K6E5A MINGW64 /c
$ cd data-engineer-challenge
```
```linux
ramse@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge
$ cd code
```
```linux
ramse@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge/code
$ cd apps
```
```linux
@DESKTOP-K6K6E5A MINGW64 /c/data-engineer-challenge/code/apps
$ docker-compose up
```
- After wait for a couple of minutes the final result of executing the above command should look like this:
- docker-compose executionlet's check the API REST, go to your browser and search for: http://localhost:8080/docs#/default
- let's try the request to the API REST
- let's check the response from the API REST
# Contributing and Feedback
Any ideas or feedback about this repository?. Help me to improve it.
# Authors
- Created by Ramses Alexander Coraspe Valdez
- Created on 2022
# License
This project is licensed under the terms of the Apache License.