https://github.com/maximlevchenko/mongodb-elasticstack-kibana-visualisation

This project showcases the use of NoSQL technologies and usage of Elastic Stack for comprehensive data processing and visualization.
https://github.com/maximlevchenko/mongodb-elasticstack-kibana-visualisation

apache-spark big-data docker docker-compose elasticsearch elasticstack logstash mongodb replication sharding visualization

Last synced: about 1 month ago
JSON representation

This project showcases the use of NoSQL technologies and usage of Elastic Stack for comprehensive data processing and visualization.

• Host: GitHub
• URL: https://github.com/maximlevchenko/mongodb-elasticstack-kibana-visualisation
• Owner: MaximLevchenko
• Created: 2024-07-25T16:31:26.000Z (4 months ago)
• Default Branch: main
• Last Pushed: 2024-07-26T12:56:20.000Z (4 months ago)
• Last Synced: 2024-10-11T22:04:35.000Z (about 1 month ago)
• Topics: apache-spark, big-data, docker, docker-compose, elasticsearch, elasticstack, logstash, mongodb, replication, sharding, visualization
• Language: JavaScript
• Homepage:
• Size: 44.9 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Project Overview

This project showcases the use of NoSQL technologies and usage of Elastic Stack for comprehensive data processing and visualization.

## Description

The project is divided into two main sections:

1. **NoSQL Technology**: Focuses on MongoDB, where sharding and replication are used to manage large datasets efficiently and ensure high availability.

2. **Elastic Stack**: Utilizes Apache Spark for data processing, with Logstash, Elasticsearch, and Kibana for indexing and visualization.

### 📝Additional information 

For detailed instructions, refer to the individual README.md files located in the respective directories. Additionally, please pay special attention to the `HOWTO.md` and `README.md` files located in the `results` folder within the Elastic Stack directory for crucial details on setting up and understanding the visualizations.

## Project Structure

The project is organized as follows:

```

/1 - NoSQL technologie

|  /data

|  /configdb

|  /db

|  |  - .gitkeep

|  |  - sample_mflix.comments.json

|  |  - sample_mflix.embedded_movies.json

|  |  - sample_mflix.movies.json

|  |  - sample_mflix.theaters.json

|  |  - sample_mflix.users.json

|  /scripts

|  |  - import_data.sh

|  |  - init-configserver.js

|  |  - init-router.js

|  |  - init-shard01.js

|  |  - init-shard02.js

|  |  - init-shard03.js

|  |  - validation_schema_init.js

|  - docker-compose.yml

|  - HOWTO.md

|  - init_script.sh

|  - queries.txt

|  - README.md

/2 - Elastic Stack   

|  /elasticsearch

|  |  /config

|  |    - elasticsearch.yml

|  /kibana

|  |  /config

|  |  | - kibana.yml

|  /logstash

|  |  /config

|  |  |  - logstash.yml

|  |  |  - pipelines.yml

|  |  /data

|  |  /pipeline

|  |  /template

|  /results

|  |  /images

|  |  - .gitkeep

|  |  - HOWTO.md

|  |  - README.md

|  /spark

|  - .gitignore

|  - docker-compose.yml

|  - HOWTO.md

|  - init_script.sh

|  - queries.txt

|  - README.md

- README.md

```