https://github.com/ahmetfurkandemir/data-engineering-tools

Data Engineering Tools
https://github.com/ahmetfurkandemir/data-engineering-tools

adminer airflow datanode flink hadoop hdfs hive hue kafka livy namenode postgresql spark

Last synced: 8 months ago
JSON representation

Data Engineering Tools

• Host: GitHub
• URL: https://github.com/ahmetfurkandemir/data-engineering-tools
• Owner: AhmetFurkanDEMIR
• License: mit
• Created: 2023-11-09T13:31:50.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-11-09T13:37:51.000Z (over 2 years ago)
• Last Synced: 2025-03-16T09:42:29.374Z (about 1 year ago)
• Topics: adminer, airflow, datanode, flink, hadoop, hdfs, hive, hue, kafka, livy, namenode, postgresql, spark
• Language: Shell
• Homepage:
• Size: 21.5 KB
• Stars: 3
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
![](https://img.shields.io/badge/Docker-2CA5E0?style=for-the-badge&logo=docker&logoColor=white) ![](https://img.shields.io/badge/Apache_Spark-FFFFFF?style=for-the-badge&logo=apachespark&logoColor=#E35A16) ![](https://img.shields.io/badge/Airflow-017CEE?style=for-the-badge&logo=Apache%20Airflow&logoColor=white) ![](https://img.shields.io/badge/Apache_Kafka-231F20?style=for-the-badge&logo=apache-kafka&logoColor=white) ![](https://img.shields.io/badge/PostgreSQL-316192?style=for-the-badge&logo=postgresql&logoColor=white)

# Data Engineering Tools

**Some important tools required for Data Engineering projects.**

Start all services

```bash

./start.sh

```

Stop all services

```bash

./stop.sh

```

Restart all services

```bash

./restart.sh

```

### **Services**

* HDFS Namenode - [http://0.0.0.0:32763](http://0.0.0.0:32763)

* Hive Server - [http://0.0.0.0:32759/](http://0.0.0.0:32759/)

* Hue - [http://0.0.0.0:32762](http://0.0.0.0:32762)

* Spark - [http://0.0.0.0:32766/](http://0.0.0.0:32766/)

    - Worker - [http://0.0.0.0:32764/](http://0.0.0.0:32764/)

* Airflow - [http://0.0.0.0:8080/](http://0.0.0.0:8080/) 

* Flink - [http://0.0.0.0:8081](http://0.0.0.0:8081)

* Livy - [http://0.0.0.0:32758](http://0.0.0.0:32758)

* Adminer - [http://0.0.0.0:32767/](http://0.0.0.0:32767/)

* PostgreSQL - 0.0.0.0:32769

* Kafka - 0.0.0.0:9092, 0.0.0.0:9093