{"id":16643413,"url":"https://github.com/sonots/fluent-plugin-record-reformer","last_synced_at":"2025-04-04T20:14:57.209Z","repository":{"id":8282433,"uuid":"9819074","full_name":"sonots/fluent-plugin-record-reformer","owner":"sonots","description":"Fluentd plugin to add or replace fields of a event record","archived":false,"fork":false,"pushed_at":"2017-09-19T08:17:20.000Z","size":136,"stargazers_count":113,"open_issues_count":2,"forks_count":25,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-01T20:45:31.420Z","etag":null,"topics":["fluentd-output-plugin","fluentd-plugin","multi-workers-ready","v10","v12","v14"],"latest_commit_sha":null,"homepage":"","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sonots.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-05-02T17:57:56.000Z","updated_at":"2024-05-19T16:17:50.000Z","dependencies_parsed_at":"2022-08-07T01:01:06.602Z","dependency_job_id":null,"html_url":"https://github.com/sonots/fluent-plugin-record-reformer","commit_stats":null,"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Ffluent-plugin-record-reformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Ffluent-plugin-record-reformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Ffluent-plugin-record-reformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sonots%2Ffluent-plugin-record-reformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sonots","download_url":"https://codeload.github.com/sonots/fluent-plugin-record-reformer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247242681,"owners_count":20907134,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluentd-output-plugin","fluentd-plugin","multi-workers-ready","v10","v12","v14"],"created_at":"2024-10-12T08:08:26.048Z","updated_at":"2025-04-04T20:14:57.191Z","avatar_url":"https://github.com/sonots.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fluent-plugin-record-reformer\n\n[![Build Status](https://secure.travis-ci.org/sonots/fluent-plugin-record-reformer.png?branch=master)](http://travis-ci.org/sonots/fluent-plugin-record-reformer)\n\nFluentd plugin to add or replace fields of a event record\n\n## Requirements\n\nSee [.travis.yml](.travis.yml)\n\nNote that `fluent-plugin-record-reformer` supports both v0.14 API and v0.12 API in one gem.\n\n## Installation\n\nUse RubyGems:\n\n    gem install fluent-plugin-record-reformer\n\n## Configuration\n\nExample:\n\n    \u003cmatch foo.**\u003e\n      type record_reformer\n      remove_keys remove_me\n      renew_record false\n      enable_ruby false\n      \n      tag reformed.${tag_prefix[-2]}\n      \u003crecord\u003e\n        hostname ${hostname}\n        input_tag ${tag}\n        last_tag ${tag_parts[-1]}\n        message ${record['message']}, yay!\n      \u003c/record\u003e\n    \u003c/match\u003e\n\nAssume following input is coming (indented):\n\n```js\nfoo.bar {\n  \"remove_me\":\"bar\",\n  \"not_remove_me\":\"bar\",\n  \"message\":\"Hello world!\"\n}\n```\n\nthen output becomes as below (indented):\n\n```js\nreformed.foo {\n  \"not_remove_me\":\"bar\",\n  \"hostname\":\"YOUR_HOSTNAME\",\n  \"input_tag\":\"foo.bar\",\n  \"last_tag\":\"bar\",\n  \"message\":\"Hello world!, yay!\",\n}\n```\n\n## Configuration (Classic Style)\n\nExample:\n\n    \u003cmatch foo.**\u003e\n      type record_reformer\n      remove_keys remove_me\n      renew_record false\n      enable_ruby false\n      tag reformed.${tag_prefix[-2]}\n      \n      hostname ${hostname}\n      input_tag ${tag}\n      last_tag ${tag_parts[-1]}\n      message ${record['message']}, yay!\n    \u003c/match\u003e\n\nThis results in same, but please note that following option parameters are reserved, so can not be used as a record key.\n\n## Option Parameters\n\n- output_tag (obsolete)\n\n    The output tag name. This option is deprecated. Use `tag` option instead\n\n- tag\n\n    The output tag name. \n\n- remove_keys\n\n    Specify record keys to be removed by a string separated by , (comma) like\n\n        remove_keys message,foo\n\n- renew_record *bool*\n\n    `renew_record true` creates an output record newly without extending (merging) the input record fields. Default is `false`.\n\n- renew\\_time\\_key *string*\n\n    `renew_time_key foo` overwrites the time of events with a value of the record field `foo` if exists. The value of `foo` must be a unix time.\n\n- keep_keys\n\n    You may want to remain some record fields although you specify `renew_record true`. Then, specify record keys to be kept by a string separated by , (comma) like\n\n        keep_keys message,foo\n\n- enable_ruby *bool*\n\n    Enable to use ruby codes in placeholders. See `Placeholders` section.\n    Default is `true` (just for lower version compatibility). \n\n- auto_typecast *bool*\n\n    Automatically cast the field types. Default is false.\n    NOTE: This option is effective only for field values comprised of a single placeholder. \n\n    Effective Examples:\n    \n        foo ${foo}\n    \n    Non-Effective Examples:\n    \n        foo ${foo}${bar}\n        foo ${foo}bar\n        foo 1\n    \n    Internally, this **keeps** the type of value if the value text is comprised of a single placeholder, otherwise, values are treated as strings. \n    \n    When you need to cast field types manually, [out_typecast](https://github.com/tarom/fluent-plugin-typecast) and [filter_typecast](https://github.com/sonots/fluent-plugin-filter_typecast) are available. \n\n## Placeholders\n\nFollowing placeholders are available:\n\n* ${record[\"key\"]} Record value of `key` such as `${record[\"message\"]}` in the above example (available from v0.8.0).\n  * Originally, record placeholders were available as `${key}` such as `${message}`. This is still kept for the backward compatibility, but would be removed in the future.\n* ${hostname} Hostname of the running machine\n* ${tag} Input tag\n* ${time} Time of the event\n* ${tags[N]} (Obsolete. Use tag\\_parts) Input tag splitted by '.'\n* ${tag\\_parts[N]} Input tag splitted by '.' indexed with N such as `${tag_parts[0]}`, `${tag_parts[-1]}`. \n* ${tag\\_prefix[N]} Tag parts before and on the index N. For example,\n\n        Input tag: prefix.test.tag.suffix\n        \n        ${tag_prefix[0]}  =\u003e prefix\n        ${tag_prefix[1]}  =\u003e prefix.test\n        ${tag_prefix[-2]} =\u003e prefix.test.tag\n        ${tag_prefix[-1]} =\u003e prefix.test.tag.suffix\n\n* ${tag\\_suffix[N]} Tag parts after and on the index N. For example,\n\n        Input tag: prefix.test.tag.suffix\n    \n        ${tag_suffix[0]}  =\u003e prefix.test.tag.suffix\n        ${tag_suffix[1]}  =\u003e test.tag.suffix\n        ${tag_suffix[-2]} =\u003e tag.suffix\n        ${tag_suffix[-1]} =\u003e suffix\n\nIt is also possible to write a ruby code in placeholders if you set `enable_ruby true` option, so you may write some codes as\n\n* ${time.strftime('%Y-%m-%dT%H:%M:%S%z')}\n* ${tag\\_parts.last}\n\nbut, please note that enabling ruby codes is not encouraged by security reasons and also in terms of the performance.\n\n## Relatives\n\nFollowing plugins look similar:\n\n* [fluent-plugin-record-modifier](https://github.com/repeatedly/fluent-plugin-record-modifier)\n* [fluent-plugin-format](https://github.com/mach/fluent-plugin-format)\n* [fluent-plugin-add](https://github.com/yu-yamada/fluent-plugin-add)\n* [filter_record_transformer](http://docs.fluentd.org/v0.12/articles/filter_record_transformer) is a Fluentd v0.12 built-in plugin which is based on record-reformer.\n\n## ChangeLog\n\nSee [CHANGELOG.md](CHANGELOG.md) for details.\n\n## Contributing\n\n1. Fork it\n2. Create your feature branch (`git checkout -b my-new-feature`)\n3. Commit your changes (`git commit -am 'Add some feature'`)\n4. Push to the branch (`git push origin my-new-feature`)\n5. Create new [Pull Request](../../pull/new/master)\n\n## Copyright\n\nCopyright (c) 2013 - 2015 Naotoshi Seo. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonots%2Ffluent-plugin-record-reformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsonots%2Ffluent-plugin-record-reformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsonots%2Ffluent-plugin-record-reformer/lists"}