Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/okushchenko/fluent-plugin-docker-journald-concat
Fluentd Filter plugin to concatenate partial log messages generated by Docker daemon with Journald logging driver
https://github.com/okushchenko/fluent-plugin-docker-journald-concat
docker fluentd fluentd-filter-plugin fluentd-plugin journald systemd
Last synced: 2 months ago
JSON representation
Fluentd Filter plugin to concatenate partial log messages generated by Docker daemon with Journald logging driver
- Host: GitHub
- URL: https://github.com/okushchenko/fluent-plugin-docker-journald-concat
- Owner: okushchenko
- License: mit
- Created: 2017-10-02T14:12:15.000Z (about 7 years ago)
- Default Branch: master
- Last Pushed: 2017-10-02T20:48:53.000Z (about 7 years ago)
- Last Synced: 2024-04-25T10:02:23.834Z (8 months ago)
- Topics: docker, fluentd, fluentd-filter-plugin, fluentd-plugin, journald, systemd
- Language: Ruby
- Size: 14.6 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-plugin-docker-journald-concat
[](https://badge.fury.io/rb/fluent-plugin-docker-journald-concat)
Fluentd Filter plugin to concatenate partial log messages generated by Docker daemon with Journald logging driver
## Requirements
| fluent-plugin-docker-journald-concat | fluentd | ruby |
|--------------------------------------|------------|--------|
| >= 0.1.3 | >= v0.14.0 | >= 2.1 |
## Installation
Add this line to your application's Gemfile:
```ruby
gem 'fluent-plugin-docker-journald-concat'
```
And then execute:
$ bundle
Or install it yourself as:
$ gem install fluent-plugin-docker-journald-concat
## Configuration
**key**
The key to be concatenated for partial events.
Default value is `"message"`.
**match\_key**
The key which indicates a partial event.
Default value is `"container_partial_message"`.
**stream\_identity\_key**
The key to determine which stream an event belongs to.
Default value is `"container_id"`.
**flush\_interval**
The number of seconds after which the last received event will be flushed.
If specified 0, wait for next line forever.
Default value is `"60"`.
**timeout\_label**
The label assigned to timed out events.
Default value is `"nil"`.
## Usage
Insert this into your Fluentd config to get started.
```aconf
@type docker_journald_concat
```
The full Fluentd config example with Systemd Journal input.
```aconf
@type systemd
tag docker
path /var/log/journal
read_from_head true
filters [{ "_TRANSPORT": "journal", "_COMM": "dockerd" }]
@type local
persistent true
path /tmp/journal.pos
field_map {
"_SOURCE_REALTIME_TIMESTAMP": "timestamp",
"_HOSTNAME": "hostname",
"PRIORITY": "priority",
"MESSAGE": "message",
"CONTAINER_NAME": "container_name",
"CONTAINER_ID": "container_id",
"CONTAINER_PARTIAL_MESSAGE": "container_partial_message"
}
field_map_strict true
@type record_modifier
# {"6" => "stdout", "3" => "stderr"}
source ${record["priority"] == "6" ? "stdout" : "stderr"}
timestamp ${record["timestamp"].to_i / 1000}
remove_keys priority
@type docker_journald_concat
@type stdout
```
You can handle timeout events and remaining buffers on shutdown of this plugin.
```aconf
@type file
path /path/to/error.log
```
Handle timed out log lines the same as normal logs.
```aconf
@type docker_journald_concat
timeout_label @NORMAL
@type relabel
@label @NORMAL
@type stdout
```
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
## License
The gem is available as open source under the terms of the [MIT License](http://opensource.org/licenses/MIT).