Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/oleksandrtimoshenko/elastic-multi-node-cluster
Creating ELK stack with multi-node Elasticsearch cluster
https://github.com/oleksandrtimoshenko/elastic-multi-node-cluster
ansible elasticsearch kibana vagrant
Last synced: 13 days ago
JSON representation
Creating ELK stack with multi-node Elasticsearch cluster
- Host: GitHub
- URL: https://github.com/oleksandrtimoshenko/elastic-multi-node-cluster
- Owner: OleksandrTimoshenko
- Created: 2024-08-20T09:23:55.000Z (4 months ago)
- Default Branch: master
- Last Pushed: 2024-10-26T16:42:19.000Z (2 months ago)
- Last Synced: 2024-11-02T23:24:46.587Z (about 2 months ago)
- Topics: ansible, elasticsearch, kibana, vagrant
- Language: Python
- Homepage:
- Size: 60.5 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: Readme.md
Awesome Lists containing this project
README
# Elastic-multi-node-cluster
## Requirements
- `ansible [core 2.14.1]`
- `Vagrant 2.4.1 (Configured)`
- `Python3, pip3`
- `GNU Make 4.3 (if you want use installation with Makefile)`
## Setup with make
- `make help`
## Or manual setup
## Install required Python libs
`pip3 install -r ./requirements.txt`
## Create vagrant VMs
`NUM_WORKERS=3 vagrant up`
## Create inventory file, generate configs from template, update /etc/hosts
`python ./setup.py`
## Start ansible provisioning
`ansible-playbook -i inventory.yaml playbook.yaml`
## Delete infrastructure
`vagrant destroy -f`
# Go to Kibana
`https:///`
### We are using self-signed sertificates, so browser will decline it
### find credentials in `./elasticsearch-master/elasticsearch_password.txt`,
## Working with data in Kibana:
```
Go to "Discover" -> "Click on existing index (if exist)" -> "Create a data view" -> setup regex for index
Create data streams
- elasticsearch
- nginx
- kibana
Import dashboards from ./kibana_dashboards folder
```
## Elasticsearch JVM setup
### elastik usually sets the best values based on the node characteristics.
### but if you know what you're doing...
1. add `bootstrap.memory_lock: true` to `/etc/elasticsearch/elasticsearch.yml`
2. create file `/etc/elasticsearch/jvm.options.d/custom.options` (see [this](https://www.elastic.co/guide/en/elasticsearch/reference/8.15/advanced-configuration.html#set-jvm-options) for more info)
```
-Xms1g
-Xmx1g
```
3. add `MAX_LOCKED_MEMORY=unlimited` to `/etc/default/elasticsearch`
4. add this lines to `/etc/security/limits.conf`
```
- nofile 65536
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited
```
5. add `session required pam_limits.so` to `/etc/pam.d/common-session`
6. Update elasticsearch service settings (add this to file):
`sudo systemctl edit elasticsearch.service`
```
[Service]
LimitMEMLOCK=infinity
```
7. Restart servive `sudo systemctl daemon-reload && sudo systemctl restart elasticsearch`