https://github.com/eslamdyab21/solar-x

SolarX is a project aiming to practice a number of different tools, from the databases and etl related work, data streaming using kafka, data processing with kafka and python, web-sockets in the nodejs backend with kafka, to sending the data to the react frontend with web-sockets, with docker and docker compose.
https://github.com/eslamdyab21/solar-x

Last synced: 9 days ago
JSON representation

SolarX is a project aiming to practice a number of different tools, from the databases and etl related work, data streaming using kafka, data processing with kafka and python, web-sockets in the nodejs backend with kafka, to sending the data to the react frontend with web-sockets, with docker and docker compose.

• Host: GitHub
• URL: https://github.com/eslamdyab21/solar-x
• Owner: eslamdyab21
• Created: 2024-06-30T14:37:36.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2025-01-07T01:12:42.000Z (24 days ago)
• Last Synced: 2025-01-07T02:24:12.675Z (24 days ago)
• Language: Python
• Homepage:
• Size: 15 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# solar-x

SolarX is a project aiming to practice a number of different tools, from the databases and etl related work, data streaming using kafka, data processing with kafka and python, web-sockets in the nodejs backend with kafka, to sending the data to the react frontend with web-sockets, with under docker and docker compose.




### Technologies used in this project

- `Python` for data processing and etl work

- `Kafka` for handling the data stream per second

- `Nodejs` and python for the backend

- `Web-socket` for the communication between the backend and the frontend

- `React` for the frontend

- `MySQL` database to store the solar and home and batteries energies per hour (updated per minute)

- `Phpmyadmin` for monitoring the database

- `Docker` and `Docker-compose`

- `Nginx` as a reverse proxy to facilitate communication




### Project Diagram

![](images/solarx3.drawio.png)




### ERD Diagram

![](images/erd-solarx.drawio.png)




### Demo

![](images/frontend3.png)

https://github.com/user-attachments/assets/3f5eab5e-ebdb-4df1-9d1b-84936c0704d8







### Env files needed for the project

1- under `mysql_database`

```env

DATABASE_USER='solarx'

DATABASE_PASSWORD='password'

DATABASE_HOST='solarx-db'

DATABASE_NAME='SolarX'

DATABASE_PORT='3306'

KAFKA_BROKER_ADDRESS="kafka:9092" 

```

2- under `kafka-websocket`

```env

BROKER_HOST='kafka'

```

3- under `frontend-gui`

Note that `localhost` will need to e changed to the host domain name or host ip 

if other devices on the host network will connect to the application

```env

REACT_APP_WS1_URL="ws://localhost:4000/ws1/"

REACT_APP_WS2_URL="ws://localhost:4000/ws2/"

REACT_APP_WS3_URL="ws://localhost:4000/ws3/"

```

4- under `solar-x` the root

```env

KAFKA_BROKER_ADDRESS="kafka:9092"

```







### A Lakehouse project for this app is currently in progress in this repo

https://github.com/eslamdyab21/SolarX-Lakehouse