https://github.com/toyama0919/fluent-plugin-file-sprintf

https://github.com/toyama0919/fluent-plugin-file-sprintf

Last synced: 3 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/toyama0919/fluent-plugin-file-sprintf
• Owner: toyama0919
• License: mit
• Created: 2014-01-12T07:57:33.000Z (over 11 years ago)
• Default Branch: master
• Last Pushed: 2017-07-28T16:17:28.000Z (almost 8 years ago)
• Last Synced: 2025-02-02T08:31:57.046Z (4 months ago)
• Language: Ruby
• Size: 15.6 KB
• Stars: 3
• Watchers: 3
• Forks: 2
• Open Issues: 4
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# fluent-plugin-file-sprintf, a plugin for [Fluentd](http://fluentd.org)

sprintf output file plugin for Fluentd.

## Installation

### td-agent(Linux)

    /usr/lib64/fluent/ruby/bin/fluent-gem install fluent-plugin-file-sprintf

### td-agent(Mac)

    sudo /usr/local/Cellar/td-agent/1.1.XX/bin/fluent-gem install fluent-plugin-file-sprintf

### fluentd only

    gem install fluent-plugin-file-sprintf

## parameter

param    |   value|exsample

--------|------|------

compress|file compress gzip(default:true)|true

buffer_path|buffer file path(require)|/tmp/buffer/test.ltsv.*.log

path|output file path(require)|/tmp/test.ltsv

format|sprintf format(require)|%s

key_names|key names comma separator(require)|ltsv

time_format|time value output format(default:%Y-%m-%d %H:%M:%S)|%Y-%m-%d %H:%M:%S

include_tag_key|tag key in record|true

tag_key_name|tag key name(default:tag)|tag_name

include_time_key|time key in record|true

time_key_name|time key name(default:time)|timestamp

rotate|rotate (default:true)|false

rotate_format|file rotate format(default:%Y%m%d)|%Y%m%d

## key_names reserved words

param    |   value|

--------|------|

time|output time string

tag|output tag string

ltsv|output ltsv string

msgpack|output msgpack string

json|output json string

## Configuration Exsample(json format)

	

		type file_sprintf

		compress true

		buffer_path /tmp/buffer/apache.json.*.log

		path /tmp/apache.json

		format %s

		key_names json

	

## Configuration Exsample(labeled tsv format)

	

		type file_sprintf

		compress true

		buffer_path /tmp/buffer/apache.ltsv.*.log

		path /tmp/apache.ltsv

		format %s

		key_names ltsv

	

## Configuration Exsample(message pack format)

	

		type file_sprintf

		compress true

		buffer_path /tmp/buffer/apache.msgpack.*.log

		path /tmp/apache.msgpack

		format %s

		key_names msgpack

	

## Configuration Exsample(custom json format)

	

		type file_sprintf

		compress true

		buffer_path /tmp/buffer/apache.json.*.log

		path /tmp/apache.json

		format {"method":"%s","agent":"%s","referer":"%s","path":"%s","host":"%s","time":"%s","tag":"%s"}

		key_names method,agent,referer,path,host,time,tag

	

## Configuration Exsample(tsv format)

	

		type file_sprintf

		compress true

		buffer_path /tmp/buffer/apache.tsv.*.log

		path /tmp/apache.tsv

		format %s\t%s\t%s\t%s\t%s\t%s\t%s

		key_names method,agent,referer,path,host,time,tag

	

## Configuration Exsample(elasticsearch bulk import format. multiline format)

	

		type file_sprintf

		compress false

		buffer_path /tmp/buffer/es.json.*.log

		path /tmp/es.json

		format { "index" : { "_index" : "test_index", "_type" : "test_type" } }\n%s

		key_names json

	

## rotate with loglotate.d exsample

```bash:/etc/logrotate.d/sprintf

/var/log/webapp.log {

  daily

  rotate 30

  create 640 td-agent td-agent

  missingok

  notifempty

  sharedscripts

  postrotate

    pid=/var/run/td-agent/td-agent.pid

    test -s $pid && kill -USR1 "$(cat $pid)"

  endscript

}

```

## Contributing

1. Fork it

2. Create your feature branch (`git checkout -b my-new-feature`)

3. Commit your changes (`git commit -am 'Add some feature'`)

4. Push to the branch (`git push origin my-new-feature`)

5. Create new Pull Request