https://github.com/josemanuel22/datalab-project
Part of my code for the Datalab project developed for the European Southern Observatory (ESO).
https://github.com/josemanuel22/datalab-project
big-data cassandra databases datalab docker elasticsearch nosql-database python
Last synced: 2 months ago
JSON representation
Part of my code for the Datalab project developed for the European Southern Observatory (ESO).
- Host: GitHub
- URL: https://github.com/josemanuel22/datalab-project
- Owner: josemanuel22
- Created: 2017-11-01T23:07:35.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2020-11-14T21:57:24.000Z (over 5 years ago)
- Last Synced: 2025-12-27T04:15:51.305Z (6 months ago)
- Topics: big-data, cassandra, databases, datalab, docker, elasticsearch, nosql-database, python
- Homepage: https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10704/107042J/Framework-to-use-modern-big-data-software-tools-to-improve/10.1117/12.2312096.short?SSO=1
- Size: 13.3 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Datalab-project
Part of my code for the Datalab project developed for the `European Southern Observatory` (ESO).
## What is inside
It contains among other things many `Dockers` with images of the installed databases (`Elasticsearch`, `Cassandra`, `Elasandra`, etc). Puppet files to automatically install docker compose. `Python` scripts, `c++` to parse, inject the data into the databases. Installation also of `Kibana`, `Grafana`, etc. and
`Dockers` with own miscellaneous applications.
Everything is a little messy.
Big Data Analysis:
* Installation Datalab for Observatory Logs: `Elasticsearch`, `Cassandra`, `Elassandra`, `Kairosdb`.
* Servers orchestration: `Puppet`.
* Installation and work with Data Analytic tools, ELK: `Jupyter`, `Grafana`, `Kibana`.
* Security: `Reverse proxy`, `openScap`, `openVAS`, `Seccubus`, `Dagda`.
* Anomaly detection: `Python` (`Pandas`, `TensorFlow`).
* Script: `Python`, `Bash`.
* Data Visualization: `Python` (`Matplotlib`, `Bokeh`, `Plotly`).
## More Info
[Conference Paper about Datalab Project](https://www.spiedigitallibrary.org/conference-proceedings-of-spie/10704/107042J/Framework-to-use-modern-big-data-software-tools-to-improve/10.1117/12.2312096.short?SSO=1)