https://github.com/logzio/fluentd-docker-logs
https://github.com/logzio/fluentd-docker-logs
integration
Last synced: about 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/logzio/fluentd-docker-logs
- Owner: logzio
- License: apache-2.0
- Created: 2021-06-02T12:03:20.000Z (about 4 years ago)
- Default Branch: main
- Last Pushed: 2025-04-06T10:53:28.000Z (2 months ago)
- Last Synced: 2025-04-06T11:29:27.570Z (2 months ago)
- Topics: integration
- Language: Python
- Homepage:
- Size: 45.9 KB
- Stars: 1
- Watchers: 4
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Fluentd-docker-logs
fluentd-docker-logs is a Docker container that uses fluentd to collect logs from other Docker containers and forward those logs to your Logz.io account.
To use this container, you'll set environment variables in your `docker run` command.
fluentd-docker-logs uses those environment variables to generate a valid fluentd configuration for the container.
fluentd-docker-logs mounts docker.sock and the Docker logs directory to the container itself, allowing fluentd to collect the logs and metadata.
fluentd-docker-logs ships logs only.
## fluentd-docker-logs setup
### 1. Pull the Docker image
Download the logzio/fluentd-docker-logs image:
```shell
docker pull logzio/fluentd-docker-logs
```
### 2. Run the container
#### Examples:
For a complete list of options, see the parameters below the code blocks.👇
#### Send to logzio listener:
```
docker run -it --rm \
--name fluentd-docker-logs \
-v $(pwd)/log:/fluentd/log \
-v /var/lib/docker/containers:/var/lib/docker/containers \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-p 5001:5001 \
-e LOGZIO_LOG_LISTENER=<> \
-e LOGZIO_LOG_SHIPPING_TOKEN=<> \
-e LOGZIO_TYPE=<> \
-e LOGZIO_INCLUDE_REGEX=<> \
logzio/fluentd-docker-logs
```
#### Via your proxy server:
```
docker run -it --rm \
--name fluentd-docker-logs \
-v $(pwd)/log:/fluentd/log \
-v /var/lib/docker/containers:/var/lib/docker/containers \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-p 5001:5001 \
-e LOGZIO_LOG_LISTENER=<> \
-e LOGZIO_LOG_SHIPPING_TOKEN=<> \
-e LOGZIO_TYPE=<> \
-e LOGZIO_PROXY_URI=<> \
-e LOGZIO_PROXY_CERT=<> \
logzio/fluentd-docker-logs
```
Use the `-p 5001:5001` flag to expose fluentd metrics in prometheus format on [http://localhost:5001/metrics](http://localhost:5001/metrics)
#### Parameters
| Parameter | Description |
| ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| LOGZIO_LOG_SHIPPING_TOKEN | **Required**. Your Logz.io account token. Replace `LOGZIO_LOG_SHIPPING_TOKEN` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. |
| LOGZIO_LOG_LISTENER | **Default**: `https://listener.logz.io:8071` .
with your region’s listener host . For more information on finding your account’s region, see Account region. |
| LOGZIO_TYPE | **Default**: `docker-fluentd`
The log type you'll use with this Docker. This is shown in your logs under the `type` field in Kibana.
Logz.io applies parsing based on `type`. |
| LOGZIO_INCLUDE_REGEX | **Default**: `.+`
Specifies regex expresion to match against container names, logs from containers that their name does not match the expresion will not be sent. |
| LOGZIO_PROXY_URI | **Default**: `nil`
Your proxy uri. |
| LOGZIO_PROXY_CERT | **Default**: `nil`
Your proxy cert. |
| ADDITIONAL_FIELDS | **Default**: `nil`
Include additional fields with every message sent, formatted as `fieldName1=fieldValue1,fieldName2=fieldValue2`. |
| LOGZIO_CONTAINER_STATUS_REGEX | **Default**: `running`
Regex to match against container status. One or more of `created`, `restarting`, `running`, `removing`, `paused`, `exited`, or `dead`. |
| LOGZIO_SLOW_FLUSH_LOG_THRESHOLD | **Default**: `20.0`
The threshold for chunk flush performance check. |
| LOGZIO_BUFFER_TYPE | **Default**: `file`
Specifies which plugin to use as the backend. |
| LOGZIO_BUFFER_PATH | **Default**: `/var/log/Fluentd-buffers/stackdriver.buffer`
Path of the buffer. |
| LOGZIO_OVERFLOW_ACTION | **Default**: `block`
Controls the behavior when the queue becomes full. Refer to [fluentd docs](https://docs.fluentd.org/output#overflow_action) for more details about your options. |
| LOGZIO_CHUNK_LIMIT_SIZE | **Default**: `2M`
Maximum size of a chunk allowed |
| LOGZIO_QUEUE_LIMIT_LENGTH | **Default**: `6`
Maximum length of the output queue. |
| LOGZIO_FLUSH_INTERVAL | **Default**: `5s`
Interval, in seconds, to wait before invoking the next buffer flush. |
| LOGZIO_RETRY_MAX_INTERVAL | **Default**: `30s`
Maximum interval, in seconds, to wait between retries. |
| LOGZIO_FLUSH_THREAD_COUNT | **Default**: `2`
Number of threads to flush the buffer. |
| LOGZIO_LOG_LEVEL | **Default**: `info`
The log level for this container. |
### 3. Check Logz.io for your logs
Spin up your Docker containers if you haven’t done so already. Give your logs a few minutes to get from your system to ours, and then open [Kibana](https://app.logz.io/#/dashboard/kibana).
### Change log
To see all avilable tags refer to this projects repo on [Docker hub](https://hub.docker.com/r/logzio/fluentd-docker-logs/tags?page=1&ordering=last_updated)
- 1.2.0:
- Bump fluentd version to `1.16`.
- 1.1.0:
- add support additional fields
- 1.0.0:
- collect and sends container logs
- Proxy support
- Expose fluentd metrics on port 5001/metrics