https://github.com/fluent-plugins-nursery/fluent-plugin-measure_time
Fluentd plugin to measure elapsed time to process messages
https://github.com/fluent-plugins-nursery/fluent-plugin-measure_time
fluentd-output-plugin fluentd-plugin to-do-v14 v10 v12
Last synced: about 1 year ago
JSON representation
Fluentd plugin to measure elapsed time to process messages
- Host: GitHub
- URL: https://github.com/fluent-plugins-nursery/fluent-plugin-measure_time
- Owner: fluent-plugins-nursery
- License: mit
- Created: 2014-04-06T09:05:00.000Z (about 12 years ago)
- Default Branch: master
- Last Pushed: 2016-05-20T13:29:36.000Z (about 10 years ago)
- Last Synced: 2025-05-13T20:15:12.525Z (about 1 year ago)
- Topics: fluentd-output-plugin, fluentd-plugin, to-do-v14, v10, v12
- Language: Ruby
- Homepage:
- Size: 38.1 KB
- Stars: 4
- Watchers: 0
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# fluent-plugin-measure_time
[](http://travis-ci.org/sonots/fluent-plugin-measure_time)
[](https://codeclimate.com/github/sonots/fluent-plugin-measure_time)
Fluentd plugin to measure elapsed time to process messages
## Installation
Use RubyGems:
gem install fluent-plugin-measure_time
## Parameters
* tag
The output tag name. Default is `measure_time`
* hook (required)
Specify the method to measure time.
* interval
The time interval to emit measurement results. Default is nil which do not compute statistics and emit the time in each measurement.
## Configuration Example 1 - Profile an Output Plugin
As an example, let's profile how long the [emit](https://github.com/sonots/fluent-plugin-grep/blob/master/lib/fluent/plugin/out_grep.rb#L56) method of [fluent-plugin-grep](https://github.com/sonots/fluent-plugin-grep) is taking.
Configure fluentd.conf as below:
For Fluentd v0.10:
```apache
type measure_time
# This makes available the `measure_time` directive for all plugins
type forward
port 24224
# measure_time plugin output comes here
type stdout
# Whatever you want to do
type stdout
type grep
add_tag_prefix greped
tag measure_time
hook emit
```
For Fluentd v0.12:
```apache
@type measure_time
# This makes available the `measure_time` directive for all plugins
@type dummy
tag raw.dummy
dummy {"message":"foo"}
# measure_time plugin output comes here
@type stdout
# Whatever you want to do
@type stdout
@type grep
add_tag_prefix greped
tag measure_time
hook emit
```
The output of fluent-plugin-measure_time will be as below:
```
measure_time: {"time":0.000849735,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
```
where `time` denotes the measured elapsed time, and `class`, `hook`, and `object_id` denotes the hooked class, the hooked method, and the object id of the plugin instance.
### interval option
fluent-plugin-measure_time outputs the elapsed time for each calling, but you can use the `interval` option when you want to get statistics in each interval.
```
measure_time: {"max":1.011,"avg":0.002","num":10,"class":"Fluent::GrepOutput","hook":"emit","object_id":83935080}
```
where `max` and `avg` are the maximum and average elapsed times, and `num` is the number of being called in each interval.
## Configuration Example (2) - Profile the in_forward plugin
I introduce an interesting example here.
Following illustration draws the sequence of that `in_forward` plugin receives a data, processes, and passes the data to output plugins.
*Sequence Diagram*
```
+–––––––––––––+ +––––––––––––––+ +––––––––––––––+
| in_forwrd | | Output | | Output |
+––––––+––––––+ +––––––+–––––––+ +––––––+–––––––+
#on_message | start = Time.now | |
+––––––––––––––––––> |
| #emit | |
| +––––––––––––––––––>
| | #emit |
| | |
| | |
| <– – – – – – – – – +
| elapsed = Time.now - start |
<– – – – – - – – – + |
| | |
+ + +
```
As the illustration, by hooking `on_message` method of `in_forward` plugin,
we can measure the blocking time taking to process the received data,
which also means that the time taking until `in_forward` will be ready for receiving a next data.
This profiling is very useful to investigate when you have a suspicion that throughputs of Fluentd fell down extremely.
The configuration will be as follows:
For Fluentd v0.10:
```apache
type measure_time
# This makes available the `measure_time` directive for all plugins
type forward
port 24224
tag measure_time
hook on_message
type stdout
# whatever you want
type stdout
```
For Fluentd v0.12:
```apache
@type measure_time
# This makes available the `measure_time` directive for all plugins
@type forward
port 24224
tag measure_time
hook on_message
@type stdout
# whatever you want
@type stdout
```
Output becomes as below:
```
measure_time: {"time":0.000849735,"class":"Fluent::ForwardInput","hook":"on_message","object_id":83935080}
```
## ChangeLog
See [CHANGELOG.md](CHANGELOG.md) for details.
## Contributing
1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new [Pull Request](../../pull/new/master)
## Copyright
Copyright (c) 2014 Naotoshi Seo. See [LICENSE](LICENSE) for details.