Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/riywo/fluent-plugin-groupcounter
https://github.com/riywo/fluent-plugin-groupcounter
Last synced: 3 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/riywo/fluent-plugin-groupcounter
- Owner: riywo
- Created: 2012-09-25T10:03:17.000Z (over 12 years ago)
- Default Branch: master
- Last Pushed: 2014-08-03T13:56:48.000Z (over 10 years ago)
- Last Synced: 2024-04-14T08:06:26.588Z (9 months ago)
- Language: Ruby
- Size: 232 KB
- Stars: 5
- Watchers: 5
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
Awesome Lists containing this project
README
# fluent-plugin-groupcounter, a plugin for [Fluentd](http://fluentd.org)
Fluentd plugin to count like SELECT COUNT(\*) GROUP BY.
## Configuration
Assume inputs are coming as followings:
apache.access: {"code":"200", "method":"GET", "path":"/index.html", "reqtime":"1.001" }
apache.access: {"code":"202", "method":"GET", "path":"/foo.html", "reqtime":"2.002" }
apache.access: {"code":"200", "method":"GET", "path":"/index.html", "reqtime":"3.003" }Think of quering `SELECT COUNT(\*) GROUP BY code,method,path`. Configuration becomes as below:
type groupcounter
aggregate tag
output_per_tag true
add_tag_prefix groupcounter
group_by_keys code,method,path
Output becomes like
groupcounter.apache.access: {"200_GET_/index.html_count":2, "202_GET_/foo.html_count":1}
## Parameters
* group\_by\_keys (semi-required)
Specify keys in the event record for grouping. `group_by_keys` or `group_by_expression` is required.
* delimiter
Specify the delimiter to join `group_by_keys`. Default is '_'.
* pattern\[1-20\]
Use `patternX` option to apply grouping more roughly. For example, adding a configuration for the above example as below
pattern1 2xx ^2\d\d
gives you an ouput like
groupcounter.apache.access: {"2xx_GET_/index.html_count":3}
* group\_by\_expression (semi-required)
Use an expression to group the event record. `group_by_keys` or `group_by_expression` is required.
For examples, for the exampled input above, the configuration as below
group_by_expression ${method}${path}/${code}
gives you an output like
groupcounter.apache.access: {"GET/index.html/200_count":1, "GET/foo.html/400_count":1}
SECRET TRICK: You can write a ruby code in the ${} placeholder like
group_by_expression ${method}${path.split(".")[0]}/${code[0]}xx
This gives an output like
groupcounter.apache.access: {"GET/index/2xx_count":1, "GET/foo/4xx_count":1}
* tag
The output tag. Default is `groupcount`.
* add\_tag\_prefix
The prefix string which will be added to the input tag. `output_per_tag yes` must be specified together.
* remove\_tag\_prefix
The prefix string which will be removed from the input tag.
* count\_interval
The interval time to count in seconds. Default is `60`.
* unit
The interval time to monitor specified an unit (either of `minute`, `hour`, or `day`).
Use either of `count_interval` or `unit`.* store\_file
Store internal data into a file of the given path on shutdown, and load on starting.
* max\_key
Specify key name in the event record to do `SELECT COUNT(\*),MAX(key_name) GROUP BY`.
For examples, for the exampled input above, adding the configuration as below
max_key reqtime
gives you an output like
groupcounter.apache.access: {"200_GET_/index.html_reqtime_max":3.003, "202_GET_/foo.html_reqtime_max":2.002}
* min\_key
Specify key name in the event record to do `SELECT COUNT(\*),MIN(key_name) GROUP BY`.
* avg\_key
Specify key name in the event record to do `SELECT COUNT(\*),AVG(key_name) GROUP BY`.
* count\_suffix
Default is `_count`
* max\_suffix
Default is `_max`. Should be used with `max_key` option.
* min\_suffix
Default is `_min`. Should be used with `min_key` option.
* avg\_suffix
Default is `_avg`. Should be used with `avg_key` option.
## ChangeLog
See [CHANGELOG.md](CHANGELOG.md) for details.
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new [Pull Request](../../pull/new/master)## Copyright
* Copyright
* Copyright (c) 2012- Ryosuke IWANAGA (riywo)
* Copyright (c) 2013- Naotoshi SEO (sonots)
* License
* Apache License, Version 2.0