Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/pblittle/docker-logstash

Docker image for Logstash 1.4
https://github.com/pblittle/docker-logstash

docker elasticsearch elk kibana logstash

Last synced: 3 months ago
JSON representation

Docker image for Logstash 1.4

Awesome Lists containing this project

README 

        
# Logstash Dockerfile



This is a highly configurable [logstash][7] (1.4.5) image running [Elasticsearch][8] (1.7.0) and [Kibana][9] (3.1.2).



## How to use this image



To run the image, you have to first decide which services you want to run in your container:



#### Full ELK stack (default)



````

$ docker run -d \

  -p 9292:9292 \

  -p 9200:9200 \

  pblittle/docker-logstash

````



#### Logstash agent only



````

$ docker run \

  pblittle/docker-logstash \

  agent

````



#### Logstash config test only



````

$ docker run \

  pblittle/docker-logstash \

  configtest

````



#### Embedded Kibana web interface only



````

$ docker run \

  pblittle/docker-logstash \

  web

````



## Logstash configuration



There are currently two supported ways of including your Logstash config files in your container:



  * Download your config files from the Internet

  * Mount a volume on the host machine containing your config files



> Any files in `/opt/logstash/conf.d` with the `.conf` extension will get loaded by logstash.



#### Download your config files from the Internet



To use your own hosted config files, your config files must be one of the following two file types:



  * A monolithic config file (`*.conf`)

  * A tarball containing your config files (`*.tar`, `*.tar.gz`, or `*.tgz`)



With your config files ready and in the correct format, set `LOGSTASH_CONFIG_URL` to your logstash config URL using the `-e` flag as follows:



    $ docker run -d \

      -e LOGSTASH_CONFIG_URL= \

      -p 9292:9292 \

      -p 9200:9200 \

      pblittle/docker-logstash



By default, if `LOGSTASH_CONFIG_URL` isn't defined, an example [logstash.conf][2] will be downloaded and used in your container.



> The default `logstash.conf` only listens on `stdin` and `file` inputs. If you wish to configure `tcp` and/or `udp` input, use your own logstash configuration files and expose the ports yourself. See [logstash documentation][10] for config syntax and more information.



#### Mount a volume containing your config files



To use config files from the local file system, mount the config directory as a volume using the `-v` flag. For example:



    $ docker run -d \

      -v :/opt/logstash/conf.d \

      -p 9292:9292 \

      -p 9200:9200 \

      pblittle/docker-logstash



## Elasticsearch server integration



If you plan on using Elasticsearch, the following three integration methods are supported:



 * A linked container running Elasticsearch

 * An external Elasticsearch server

 * The embedded Elasticsearch server



#### Linked container running Elasticsearch



If you want to link to a container running Elasticsearch, simply use the `--link` flag to connect to the container:



    $ docker run -d \

      --link :es \

      -p 9292:9292 \

      pblittle/docker-logstash



To have the linked Elasticsearch container's `bind_host` and `port` automatically detected, you will need to set the `bind_host` and `port` to `ES_HOST` and `ES_PORT` respectively in your elasticsearch output config. For example:



    output {

      elasticsearch {

        bind_host => "ES_HOST"

        port => "ES_PORT"

        protocol => "http"

      }

    }



If you are linking to an Elasticsearch container running on `172.0.4.20:9200`, the config above will be transformed into:



    output {

      elasticsearch {

        host => "172.0.4.20"

        port => "9200"

        protocol => "http"

      }

    }



#### External Elasticsearch server



If you are using an external Elasticsearch server, simply set the `ES_HOST` and `ES_PORT` environment variables in your `run` command:



    $ docker run -d \

      -e ES_HOST= \

      -e ES_PORT= \

      -p = \

      -p 9292:9292 \

      pblittle/docker-logstash



#### Embedded Elasticsearch server



The embedded Elasticsearch server will be used by default if you don't provide either of the configuration options above.



> Please note, the embedded Elasticsearch server was not designed for use in Production.



To make the data directory persistent, you can bind mount it with the following argument for `docker run`:



    -v $PWD/data:/opt/logstash/data



## Optional, build and run the image from source



If you prefer to build from source rather than use the [pblittle/docker-logstash][1] trusted build published to the public Docker Registry, execute the following:



    $ git clone https://github.com/pblittle/docker-logstash.git

    $ cd docker-logstash



> If you are using [Vagrant][3], you can build and run the container in a VM by executing:

>

>     $ vagrant up

>     $ vagrant ssh

>     $ cd /vagrant/1.4



From there, build and run a container using the newly created virtual machine:



    $ make



## Finally, verify the installation



You can now verify the logstash installation by visiting the sample Kibana dashboard:



    http://:9292/index.html#/dashboard/file/default.json



## Thank you



A huge thank you to the project [Contributors][4] and users. I really appreciate the support.



## Contributing



1. Fork it

2. Checkout the develop branch (`git checkout -b develop`)

3. Create your feature branch (`git checkout -b my-new-feature`)

4. Commit your changes (`git commit -am 'Add some feature'`)

5. Push to the branch (`git push origin my-new-feature`)

6. Create new Pull Request



## License



This application is distributed under the [Apache License, Version 2.0][5].



[1]: https://registry.hub.docker.com/u/pblittle/docker-logstash

[2]: https://gist.githubusercontent.com/pblittle/8778567/raw/logstash.conf

[3]: https://www.vagrantup.com

[4]: https://github.com/pblittle/docker-logstash/graphs/contributors

[5]: http://www.apache.org/licenses/LICENSE-2.0

[7]: https://www.elastic.co/products/logstash

[8]: https://www.elastic.co/products/elasticsearch

[9]: https://www.elastic.co/products/kibana

[10]: https://www.elastic.co/guide/en/logstash/current/configuration.html