https://github.com/fluent-plugins-nursery/fluent-plugin-stats

Fluentd plugin to calculate statistics such as sum, max, min, avg.
https://github.com/fluent-plugins-nursery/fluent-plugin-stats

fluentd-output-plugin fluentd-plugin v10 v12 v14

Last synced: about 1 year ago
JSON representation

Fluentd plugin to calculate statistics such as sum, max, min, avg.

• Host: GitHub
• URL: https://github.com/fluent-plugins-nursery/fluent-plugin-stats
• Owner: fluent-plugins-nursery
• License: mit
• Created: 2013-05-05T17:07:33.000Z (about 13 years ago)
• Default Branch: master
• Last Pushed: 2017-09-14T19:33:30.000Z (over 8 years ago)
• Last Synced: 2025-05-15T13:22:47.173Z (about 1 year ago)
• Topics: fluentd-output-plugin, fluentd-plugin, v10, v12, v14
• Language: Ruby
• Homepage:
• Size: 43 KB
• Stars: 10
• Watchers: 3
• Forks: 4
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • Changelog: CHANGELOG.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# fluent-plugin-stats [![Build Status](https://secure.travis-ci.org/sonots/fluent-plugin-stats.png?branch=master)](http://travis-ci.org/sonots/fluent-plugin-stats)

Fluentd plugin to calculate statistics such as sum, max, min, avg.

## Configuration

### Example 1

Get sum for xxx\_count, max for xxx\_max, min for xxx\_min, avg for xxx\_avg

    

      type stats

      interval 5s

      add_tag_prefix stats

      sum .*_count$

      max .*_max$

      min .*_min$

      avg .*_avg$

    

Assuming following inputs are coming:

    foo.bar: {"4xx_count":1,"5xx_count":2","reqtime_max":12083,"reqtime_min":10,"reqtime_avg":240.46}

    foo.bar: {"4xx_count":4,"5xx_count":2","reqtime_max":24831,"reqtime_min":82,"reqtime_avg":300.46}

then output bocomes as belows: 

    stats.foo.bar: {"4xx_count":5,"5xx_count":4","reqtime_max":24831,"reqtime_min":10,"reqtime_avg":270.46}

### Example 2

Get sum, max, min, avg for the same key

    

      type stats

      interval 5s

      add_tag_prefix stats

      sum ^reqtime$

      max ^reqtime$

      min ^reqtime$

      avg ^reqtime$

      sum_suffix _sum

      max_suffix _max

      min_suffix _min

      avg_suffix _avg

    

Assuming following inputs are coming:

    foo.bar: {"reqtime":1.000}

    foo.bar: {"reqtime":2.000}

then output bocomes as belows: 

    stats.foo.bar: {"reqtime_sum":3.000,"reqtime_max":2.000,"reqtime_min":1.000,"reqtime_avg":1.500}

## Parameters

- sum, min, max, avg

    Target of calculation. Specify input keys by a regular expression

- sum\_keys, min\_keys, max\_keys, avg\_keys

    Target of calculation. Specify input keys by a string separated by , (comma) such as

        sum_keys 4xx_count,5xx_count

- sum\_suffix, min\_suffix, max\_suffix, avg\_suffix

    Add a suffix to keys of the output record

- interval

    The interval to calculate in seconds. Default is 5s. 

- tag

    The output tag name. Required for aggregate `all`. 

- add_tag_prefix

    Add tag prefix for output message. Default: 'stats'

- remove_tag_prefix

    Remove tag prefix for output message. 

- add_tag_suffix

    Add tag suffix for output message.

- remove_tag_suffix

    Remove tag suffix for output message. 

* remove_tag_slice *min..max*

    Remove tag parts by slice function. FYI: This option behaves like `tag.split('.').slice(min..max)`.

    For example,

        remove_tag_slice 0..-2

    changes an input tag `foo.bar.host1` to `foo.bar`.

* aggregate

    Aggregation unit. One of `all`, `in_tag`, `out_tag` can be specified. Default is `in_tag`.

    * `all` calculate stats for all input messages and emit one message in each interval.

    * `in_tag` calculate stats for each input tag seperately.

    * `out_tag` calculate stats for for each tag *modified* by `add_tag_prefix`, `remove_tag_prefix`, or `remove_tag_slice`.

- store_file

    Store internal data into a file of the given path on shutdown, and load on starting.

- zero_emit

    Emit 0 on the next interval. This is useful for some software which requires to reset data such as [GrowthForecast](http://kazeburo.github.io/GrowthForecast/) . 

        stats.foo.bar: {"4xx_count":5,"5xx_count":4","reqtime_max":24831,"reqtime_min":10,"reqtime_avg":270.46}

        # after @interval later

        stats.foo.bar: {"4xx_count":0,"5xx_count":0","reqtime_max":0,"reqtime_min":0,"reqtime_avg":0}

## ChangeLog

See [CHANGELOG.md](CHANGELOG.md) for details.

## ToDo

Get the number of denominator to calculate `avg` from input json field. 

## Contributing

1. Fork it

2. Create your feature branch (`git checkout -b my-new-feature`)

3. Commit your changes (`git commit -am 'Add some feature'`)

4. Push to the branch (`git push origin my-new-feature`)

5. Create new [Pull Request](../../pull/new/master)

## Copyright

Copyright (c) 2013 Naotoshi Seo. See [LICENSE](LICENSE) for details.