{"id":21056916,"url":"https://github.com/fluent-plugins-nursery/fluent-plugin-grepcounter","last_synced_at":"2025-05-15T23:33:31.317Z","repository":{"id":7359969,"uuid":"8684881","full_name":"fluent-plugins-nursery/fluent-plugin-grepcounter","owner":"fluent-plugins-nursery","description":"Fluentd plugin to count the number of matched messages, and emit if exeeds the threshold","archived":false,"fork":false,"pushed_at":"2017-01-23T06:47:55.000Z","size":74,"stargazers_count":19,"open_issues_count":3,"forks_count":10,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-15T16:29:40.224Z","etag":null,"topics":["fluentd-output-plugin","fluentd-plugin","to-do-multi-workers-ready","to-do-v14","v10","v12"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluent-plugins-nursery.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-03-10T12:29:10.000Z","updated_at":"2024-05-19T12:47:05.000Z","dependencies_parsed_at":"2022-09-07T03:00:33.253Z","dependency_job_id":null,"html_url":"https://github.com/fluent-plugins-nursery/fluent-plugin-grepcounter","commit_stats":null,"previous_names":["sonots/fluent-plugin-grepcounter"],"tags_count":20,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent-plugins-nursery%2Ffluent-plugin-grepcounter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent-plugins-nursery%2Ffluent-plugin-grepcounter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent-plugins-nursery%2Ffluent-plugin-grepcounter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent-plugins-nursery%2Ffluent-plugin-grepcounter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluent-plugins-nursery","download_url":"https://codeload.github.com/fluent-plugins-nursery/fluent-plugin-grepcounter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254442429,"owners_count":22071864,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluentd-output-plugin","fluentd-plugin","to-do-multi-workers-ready","to-do-v14","v10","v12"],"created_at":"2024-11-19T16:54:56.093Z","updated_at":"2025-05-15T23:33:26.302Z","avatar_url":"https://github.com/fluent-plugins-nursery.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fluent-plugin-grepcounter\n\n[![Build Status](https://secure.travis-ci.org/sonots/fluent-plugin-grepcounter.png?branch=master)](http://travis-ci.org/sonots/fluent-plugin-grepcounter)\n[![Coverage Status](https://coveralls.io/repos/sonots/fluent-plugin-grepcounter/badge.png?branch=master)](https://coveralls.io/r/sonots/fluent-plugin-grepcounter?branch=master)\n\nFluentd plugin to count the number of matched messages, and emit if exeeds the `threshold`. \n\n## Configuration\n\nAssume inputs from another plugin are as belows:\n\n    syslog.host1: {\"message\":\"20.4.01/13T07:02:11.124202 INFO GET /ping\" }\n    syslog.host1: {\"message\":\"20.4.01/13T07:02:13.232645 WARN POST /auth\" }\n    syslog.host1: {\"message\":\"20.4.01/13T07:02:21.542145 WARN GET /favicon.ico\" }\n    syslog.host1: {\"message\":\"20.4.01/13T07:02:43.632145 WARN POST /login\" }\n\nAn example of grepcounter configuration:\n\n    \u003cmatch syslog.**\u003e\n      type grepcounter\n      count_interval 60\n      input_key message\n      regexp WARN\n      exclude favicon.ico\n      threshold 1\n      add_tag_prefix warn.count\n    \u003c/match\u003e\n\nThen, output bocomes as belows (indented):\n\n    warn.count.syslog.host1: {\n      \"count\":2,\n      \"message\":[\"20.4.01/13T07:02:13.232645 WARN POST /auth\",\"20.4.01/13T07:02:43.632145 WARN POST /login\"],\n      \"input_tag\":\"syslog.host1\",\n      \"input_tag_last\":\"host1\",\n    }\n\n### Output message by joining with a delimiter\n\nAs default, the `grepcounter` plugin outputs matched `message` as an array as shown above. \nYou may want to output `message` as a string, then use `delimiter` option like:\n\n    \u003cmatch syslog.**\u003e\n      type grepcounter\n      count_interval 60\n      input_key message\n      regexp WARN\n      exclude favicon.ico\n      threshold 1\n      add_tag_prefix warn.count\n      delimiter \\n\n    \u003c/match\u003e\n\nThen, output bocomes as belows (indented). You can see the `message` field is joined with \\n.\n\n    warn.count.syslog.host1: {\n      \"count\":2,\n      \"message\":\"20.4.01/13T07:02:13.232645 WARN POST /auth\\n20.4.01/13T07:02:43.632145 WARN POST /login\",\n      \"input_tag\":\"syslog.host1\",\n      \"input_tag_last\":\"host1\",\n    }\n\n## Parameters\n\n- count\\_interval\n\n    The interval time to count in seconds. Default is 60.\n\n- input\\_key *field\\_key*\n\n    The target field key to grep out. Use with regexp or exclude. \n\n- regexp *regexp*\n\n    The filtering regular expression\n\n- exclude *regexp*\n\n    The excluding regular expression like grep -v\n\n- regexp[1-20] *field\\_key* *regexp* (experimental)\n\n    The target field key and the filtering regular expression to grep out. No `message` is outputted in this case.\n\n- exclude[1-20] *field_key* *regexp* (experimental)\n\n    The target field key and the excluding regular expression like grep -v. No `message` is outputted in this case.\n\n- threshold\n\n    The threshold number to emit. Emit if `count` value \u003e= specified value.\n\n- greater\\_equal\n\n    This is same with `threshold` option. Emit if `count` value is greater than or equal to (\u003e=) specified value. \n    \n- greater\\_than\n\n    Emit if `count` value is greater than (\u003e) specified value. \n    \n- less\\_than\n\n    Emit if `count` value is less than (\u003c) specified value. \n\n- less\\_equal\n\n    Emit if `count` value is less than or equal to (\u003c=) specified value. \n\n- tag\n\n    The output tag. Required for aggregate `all`. \n\n- add\\_tag\\_prefix\n\n    Add tag prefix for output message\n\n- remove\\_tag\\_prefix\n\n    Remove tag prefix for output message\n\n- add\\_tag\\_suffix\n\n    Add tag suffix for output message\n\n- remove\\_tag\\_suffix\n\n    Remove tag suffix for output message\n\n* remove_tag_slice *min..max*\n\n    Remove tag parts by slice function. FYI: This option behaves like `tag.split('.').slice(min..max)`.\n\n    For example,\n\n        remove_tag_slice 0..-2\n\n    changes an input tag `foo.bar.host1` to `foo.bar`. \n\n* aggregate\n\n    Aggregation unit. One of `all`, `in_tag`, `out_tag` can be specified. Default is `all`.\n\n    * `all` counts summation for all input messages and emit one message in each interval.\n    * `in_tag` counts summation for each input tag seperately. \n    * `out_tag` counts summation for each tag *modified* by `add_tag_prefix`, `remove_tag_prefix`, or `remove_tag_slice`. \n\n- delimiter\n\n    Output matched messages after `join`ed with the specified delimiter.\n\n- replace\\_invalid\\_sequence\n\n    Replace invalid byte sequence in UTF-8 with '?' character if `true`\n\n- store\\_file\n\n    Store internal count data into a file of the given path on shutdown, and load on statring. \n\n## ChangeLog\n\nSee [CHANGELOG.md](CHANGELOG.md) for details.\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new [Pull Request](../../pull/new/master)\n\n## Copyright\n\nCopyright (c) 2013 Naotoshi SEO. See [LICENSE](LICENSE) for details.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent-plugins-nursery%2Ffluent-plugin-grepcounter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluent-plugins-nursery%2Ffluent-plugin-grepcounter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent-plugins-nursery%2Ffluent-plugin-grepcounter/lists"}