https://github.com/julianhm9612/docker-elk-stack
Example of an elastic stack configuration in docker :whale:
https://github.com/julianhm9612/docker-elk-stack
docker docker-compose elasticsearch filebeat kibana logstash
Last synced: about 2 months ago
JSON representation
Example of an elastic stack configuration in docker :whale:
- Host: GitHub
- URL: https://github.com/julianhm9612/docker-elk-stack
- Owner: Julianhm9612
- License: mit
- Created: 2022-09-29T20:18:00.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2023-05-16T16:54:19.000Z (about 3 years ago)
- Last Synced: 2025-04-05T21:46:48.547Z (about 1 year ago)
- Topics: docker, docker-compose, elasticsearch, filebeat, kibana, logstash
- Homepage:
- Size: 940 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# docker-elk-stack
## Introduction
Example of an elastic stack configuration in docker :whale:
## Work flow
> log -> filebeat -> logstash -> elasticsearch <- kibana
## Main Features
- Security enabled by default.
- Example of reading data from log file.
- Example of reading postgresql data.
## Requirements
- [Docker 20.05 or higher](https://docs.docker.com/install/)
- [Docker-Compose 1.29 or higher](https://docs.docker.com/compose/install/)
- 4GB RAM (For Windows and MacOS make sure Docker's VM has more than 4GB+ memory.)
## Initial configuration
### 1. Run elastic search container
> docker run -d --name elasticsearch elasticsearch:8.4.3
### 2. Running an Interactive Shell in a elastic search Container
> docker exec -it elasticsearch sh
### 3. Create a directory called certs and enter
> mkdir certs && cd certs
### 4. Enable SSL
Create a self-signed certificate for Elasticsearch
> /usr/share/elasticsearch/bin/elasticsearch-certutil ca
> /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12 --dns elasticsearch,logstash
> /usr/share/elasticsearch/bin/elasticsearch-certutil cert --pem -ca elastic-stack-ca.p12 --dns kibana
> openssl pkcs12 -in elastic-certificates.p12 -out logstash.pem -clcerts -nokeys
> openssl pkcs12 -in elastic-certificates.p12 -nocerts -nodes | sed -ne '/-BEGIN PRIVATE KEY-/,/-END PRIVATE KEY-/p' > logstash-ca.key
> openssl pkcs12 -in elastic-certificates.p12 -cacerts -nokeys -chain | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > logstash-ca.crt
> openssl pkcs12 -in elastic-certificates.p12 -clcerts -nokeys | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > logstash.crt
> /usr/share/elasticsearch/bin/elasticsearch-certutil cert --ca-cert logstash-ca.crt --ca-key logstash-ca.key --dns logstash --pem
> openssl pkcs8 -in logstash-ca.key -topk8 -nocrypt -out logstash.pkcs8.key
### 5. Get out of the container
> exit
### 6. Extract elk certificates from docker
> docker cp elasticsearch:/usr/share/elasticsearch/certs ./certs
### 7. Enable SSL and TLS
> /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password
> /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
> /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
> /usr/share/elasticsearch/bin/elasticsearch-keystore add xpack.security.http.ssl.truststore.secure_password
### 8. Create passwords for basic users
Run the next command to generate passwords for all the built-in users:
> bin/elasticsearch-setup-passwords auto
## Customize Config
- templates/custom-logs.template.json : Change it to your log index
```
# Make your own log index
{
...
"mappings": {
"properties": {
"name": {
"type": "keyword"
},
"class": {
"type": "keyword"
},
"state": {
"type": "integer"
},
"@timestamp": {
"type": "date"
}
}
}
}
```
- logstash.conf
```
# Change 'timestamp' to your log custom timestamp key
filter {
...
date{
match => ["timestamp", "UNIX_MS"]
target => "@timestamp"
}
}
```
```
# Change 'time.localtime' to your location time
filter {
...
ruby {
code => "event.set('indexDay', event.get('[@timestamp]').time.localtime('+09:00').strftime('%Y%m%d'))"
}
}
```
## Usage
To run the entire stack
> docker-compose up
To down the stack
> docker-compose down
## if you have errors in chrome with the certificate you can run
sendCommand(SecurityInterstitialCommandId.CMD_PROCEED)
## Urls and ports
### Kibana
https://localhost:5601/
### Elasticsearch
https://localhost:9200/
https://localhost:9200/_cluster/health/?pretty
https://localhost:9200/_xpack
https://localhost:9200/_cat/indices?v
https://localhost:9200/_aliases
### Filebeat
http://localhost:5066/?pretty
# Task List
- [] Automatic self-signed certificate generation
- []
- []
# License
[MIT License](https://raw.githubusercontent.com/julianhm9612/docker-elk-stack/master/LICENSE)
Copyright (c) 2023 Julian Henao Marin
# Contribution
PR(s) are Open and Welcomed.