{"id":41774290,"url":"https://github.com/nom3ad/fluent-plugin-keyvalue-parser","last_synced_at":"2026-01-25T03:23:06.862Z","repository":{"id":56847054,"uuid":"79724510","full_name":"nom3ad/fluent-plugin-keyvalue-parser","owner":"nom3ad","description":"Fluent parser plugin for key-value formatted logs.","archived":false,"fork":false,"pushed_at":"2017-05-17T15:40:52.000Z","size":21,"stargazers_count":5,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-14T10:41:19.515Z","etag":null,"topics":["elk","fluentd","fluentd-input-plugin","fluentd-plugin","parser-plugin","td-agent"],"latest_commit_sha":null,"homepage":"https://rubygems.org/gems/fluent-plugin-keyvalue-parser","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nom3ad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-22T15:03:53.000Z","updated_at":"2024-06-05T10:18:05.000Z","dependencies_parsed_at":"2022-09-09T01:00:46.036Z","dependency_job_id":null,"html_url":"https://github.com/nom3ad/fluent-plugin-keyvalue-parser","commit_stats":null,"previous_names":["arunmj/fluent-plugin-keyvalue-parser"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/nom3ad/fluent-plugin-keyvalue-parser","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nom3ad%2Ffluent-plugin-keyvalue-parser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nom3ad%2Ffluent-plugin-keyvalue-parser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nom3ad%2Ffluent-plugin-keyvalue-parser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nom3ad%2Ffluent-plugin-keyvalue-parser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nom3ad","download_url":"https://codeload.github.com/nom3ad/fluent-plugin-keyvalue-parser/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nom3ad%2Ffluent-plugin-keyvalue-parser/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28742974,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-25T02:46:29.005Z","status":"ssl_error","status_checked_at":"2026-01-25T02:44:29.968Z","response_time":113,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elk","fluentd","fluentd-input-plugin","fluentd-plugin","parser-plugin","td-agent"],"created_at":"2026-01-25T03:23:06.220Z","updated_at":"2026-01-25T03:23:06.850Z","avatar_url":"https://github.com/nom3ad.png","language":"Ruby","readme":"# fluent-plugin-keyvalue-parser\n\n[Fluent](http://www.fluentd.org/) parser plugin for key:value formatted logs.\n\n\n## Installation\n\n```shell\n$ td-agent-gem install fluent-plugin-keyvalue-parser\n```\n\n## How to use\n\nEdit `/etc/td-agent/td-agent.conf` file.\n\n* with tail plugin\n```conf\n\u003csource\u003e\n  type tail\n  path /var/log/netscreen.log\n  tag  netscreen_logs \n  pair_delimiter  \",\"\n  key_value_seperator \"=\"\n  pos_file /var/run/td-agent/netscreen-log.pos\n  format keyvalue\n\u003c/source\u003e\n```\n* with parser plugin\n```conf\n\u003cfilter tag\u003e\n type parser\n format keyvalue\n pair_delimiter  \",\"\n key_value_seperator \"=\"\n key_name keyToParse\n\u003c/filter\u003e\n```\nusing above configuration,\n```\nkey1=val1,key2=value2,\"some key\" = somevalue,diff_key=\"another value\"\n```\nwill be parsed as\n\n```json\n{\"key1\":\"val1\", \"key2\":\"value2\",\"some key\":\"somevalue\",\"diff_key\":\"another value\"}\n```\n\n#### NOTE\n* if the key is not in quotes and pair_delimiter occures in key,plugin will handle it.\n  \n  eg:\n  \n  In below log, *pair_delimiter = \" \"  (space)*  is occured in key 'src zone'. \n    \n  `devname=FT6H duration=194 service=http proto=6 `**`src zone=Trust`**` port=40055 policy_id=194`\n  \n  will be parsed as \n  ```json\n  {\"devname\":\"FT6H\", \"duration\":\"194\",\"service\":\"http\",\"src zone\":\"Trust\",\"policy_id\":\"194\"}\n  ```\n* But if value is not quoted, you should use optional parameter *'adjustment_rules'* to correct the parsing.\n\n## Option Parameters\n\n- **pair_delimiter**\n    \n    delimiter which seperate each key-value pairs. can be multi-character.\n    whitespaces or tabs can be given in quotes: ie, \" \" or \"\\t\" .\n    By default it is \",\".     \n\n- **key_value_seperator**\n\n    A string or character that seprates key and its value.\n    By default it is \"=\"\n- **adjustment_rules**\n\n    Regular expression rules for some keys, represented as json , to adjust parsed records accordingly.\n    \n    {key1:regex1,key2:regex2}\n    \n   eg:\n   \n   normally following logs,\n   \n  `devname=FT6H `**`service=http`**`proto=6 src zone=Trust dst zone=Untrust`\n  \n  `devname=FT6H `**`service=NETBIOS (NS)`**`proto=17 src zone=Trust dst zone=Untrust`\n  \n   will be parsed as \n   \n   ```json\n   {\"devname\":\"FT6H\",\"service\":\"http\",\"proto\":\"6\",\"src zone\":\"Trust\",\"dst zone\":\"Untrust\"}\n   \n   {\"devname\":\"FT6H\",\"service\":\"NETBIOS\",\"(NS) proto\":\"6\",\"src zone\":\"Trust\",\"dst zone\":\"Untrust\"}\n   ```   \n   in second case, key *\"service\"* only received first part of its value, becouse value not quoted and delimiter(here space) occured in the value. \n   \n   Also next key *\"proto\"* is wrongly parsed as *\"(NS) proto\"*.\n   \n   to rectify this problem, we can use,\n   \n   `adjustment_rules {\"service\":\"NETBIOS \\\\(.*\\\\)\"}`  in configuration.\n   \n   this will parse *service* key with a value containing *NETBIOS (NS)* whenever it occures.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnom3ad%2Ffluent-plugin-keyvalue-parser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnom3ad%2Ffluent-plugin-keyvalue-parser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnom3ad%2Ffluent-plugin-keyvalue-parser/lists"}