https://github.com/LukasLoeffler/data-graph
Flow and event based data processing
https://github.com/LukasLoeffler/data-graph
data-processing etl etl-pipeline flow-based-programming graph graphical-user-interface low-code no-code
Last synced: 27 days ago
JSON representation
Flow and event based data processing
- Host: GitHub
- URL: https://github.com/LukasLoeffler/data-graph
- Owner: LukasLoeffler
- Created: 2020-11-03T19:08:21.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-06-13T07:35:18.000Z (almost 3 years ago)
- Last Synced: 2024-11-10T22:40:10.668Z (6 months ago)
- Topics: data-processing, etl, etl-pipeline, flow-based-programming, graph, graphical-user-interface, low-code, no-code
- Language: Vue
- Homepage:
- Size: 3.86 MB
- Stars: 13
- Watchers: 2
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# data-graph
Application to showcase concepts and ideas for event-based data processing.
It incorporates concepts of NodeRED and ApacheNiFi and focuses on user-interaction and user-feedback to improve the users understanding of the data flow.
Features:
* Realtime traffic indication
* Realtime error indication
* Multiple node inputs/outputs
* Interactive node editing
* Focus on usability and UX
## Backend
Data processing backend written in JavaScript/TypeScript executed by NodeJS.
## Frontend
Written in VueJS.
Node-Editor based on [BaklavaJS](https://github.com/newcat/baklavajs).
# Deployment
The application is fully dockerized and ca be started with ```docker-compose up```.
* The frontend and the backend are initially built.
* Redis and MongoDB are pulled from DockerHub.
# Development
NodeJS > v12 required and npm required.
Installation of frontend:
```
cd frontend
npm install
```
Installation of backend:
```
cd backend
npm install
```
Running local instances of redis and mongo-db on default ports:
* Redis: 6379
* Mongo: 27017
For startup of both backend and frontend npm scripts are provided in respective packacke.json