{"id":28501951,"url":"https://github.com/fluent/fluent-plugin-sanitizer","last_synced_at":"2025-07-05T02:31:49.866Z","repository":{"id":43870557,"uuid":"333262893","full_name":"fluent/fluent-plugin-sanitizer","owner":"fluent","description":null,"archived":false,"fork":false,"pushed_at":"2022-12-21T14:57:25.000Z","size":38,"stargazers_count":15,"open_issues_count":2,"forks_count":7,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-06-08T16:08:26.917Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluent.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-27T01:05:48.000Z","updated_at":"2025-01-21T18:20:44.000Z","dependencies_parsed_at":"2023-01-30T04:00:34.330Z","dependency_job_id":null,"html_url":"https://github.com/fluent/fluent-plugin-sanitizer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fluent/fluent-plugin-sanitizer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-sanitizer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-sanitizer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-sanitizer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-sanitizer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluent","download_url":"https://codeload.github.com/fluent/fluent-plugin-sanitizer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluent%2Ffluent-plugin-sanitizer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263671745,"owners_count":23494027,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-08T16:08:32.808Z","updated_at":"2025-07-05T02:31:49.861Z","avatar_url":"https://github.com/fluent.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fluent-plugin-sanitizer\nSanitizeris [Fluentd](https://fluentd.org/) filter plugin to mask sensitive information. With Sanitizer, you can mask based on key-value pairs on the fly in between Fluentd processes. Sanitizer provides options which enable you to mask values with custom rules. In custom rules, you can specify patterns such as IP addresses, hostnames in FQDN style, regular expressions and keywords. In terms of IP addresses and hostnames, Sanitizer delivers useful options which allows you to easily mask IP addresses and hostnames in complex messages.\n\n## Installation\nWhen you are using OSS Fluentd :\n```\nfluent-gem install fluent-plugin-sanitizer\n```\nWhen you are using td-agent :\n```\ntd-agent-gem install fluent-plugin-sanitizer\n```\n\n## Configuration\n### Parameters\n- hash_salt (optional) : hash salt used when calculating hash value with original information. \n- hash_scheme (optional) : Hash scheme to use for generating hash value. Supported schemes are `md5`,`sha1`,`sha256`,`sha384` and `sha512`. (default: `md5`)\n- rule options : \n  - keys (mandatory) :  Name of keys whose values will be masked. You can specify multiple keys. When keys are nested, you can use {parent key}.{child key} like \"kubernetes.master_url\". \n  - pattern_ipv4 (optional)  : Mask IP addresses in IPv4 format. You can use “true” or “false”. (defalt: false)\n  - pattern_fqdn (optional)  : Mask hostname in FQDN style. You can use “true” or “false”. (defalt: false)\n  - pattern_regex (optional)  : Mask value mactches custom regular expression.\n    - regex_capture_group (optional) : If you define capture group in regular expression, you can specify the name of capture group to be masked.\n    - pattern_regex_prefix (optional) : Define prefix used for masking vales. (default: Regex)\n  - pattern_keywords (optional)  : Mask values match custom keywords. You can specify multiple keywords. \n    - pattern_keywords_prefix (optional) : Define prefix used for masking vales. (default: Keyword)\n\nYou can specify multiple rules in a single configuration. It is also possible to define multiple pattern options in a single rule like the following sample.\n  \n```\n\u003cfilter **\u003e\n  @type sanitizer\n  hash_salt mysalt\n  \u003crule\u003e\n    keys source, kubernetes.master_url\n    pattern_ipv4 true\n  \u003c/rule\u003e\n  \u003crule\u003e\n    keys hostname, host\n    pattern_fqdn true\n  \u003c/rule\u003e\n  \u003crule\u003e\n    keys message, system.log\n    pattern_regex /^Hello World!$/\n    pattern_keywords password, passwd\n  \u003c/rule\u003e\n\u003c/filter\u003e\n```\n\n## Use cases\n### Mask IP addresses and Hostnames\nMasking IP addresses and hostnames is one of the typical use cases of security operations. You just need to specify the name of keys that potentially  have IP addresses and hostnames in value. Here is a configuration sample as well as input and output samples. \n\n**Configuration sample**\n```\n\u003cfilter **\u003e\n  @type sanitizer\n  hash_salt mysalt\n  hash_scheme md5\n  \u003crule\u003e\n    keys ip\n    pattern_ipv4 true\n  \u003c/rule\u003e\n  \u003crule\u003e\n    keys host\n    pattern_fqdn true\n  \u003c/rule\u003e\n  \u003crule\u003e\n    keys system.url, system.log\n    pattern_ipv4 true\n    pattern_fqdn true\n  \u003c/rule\u003e\n\u003c/filter\u003e\n```\n**Input sample**\n```\n {\n     \"ip\" : \"192.168.10.10\",\n     \"host\" : \"test01.demo.com\",\n      \"system\" : {\n         \"url\" : \"https://test02.demo.com:8000/event\",\n         \"log\" : \"access from 192.168.10.100 was blocked\"\n     }\n  }\n```\n**Output sample**\n```\n {\n     \"ip\" : \"IPv4_94712b06963e277fe28469388323665d\",\n     \"host\" : \"FQDN_37de34e3d799de477c742d8d7bb35550\",\n     \"system\" : {\n         \"url\" : \"https://FQDN_e9a59624f555d02f06209c9942dded19:8000/event\"\n         \"log\" : \"access from IPv4_f7374d61e6d21dc1105f70358a5f8e8f was blocked\"\n     }\n }\n```\n### Mask words match custom keyword and regular expression\nIn case log messages including sensitive information such as SSN and phone number, Sanitizer could also help you. If you know the exact keyword that needs to be masked, you can use the keyword option. You can also use the regex option if you like to mask information which matches custom a regular expression.\n\n**Configuration sample**\n```\n\u003cfilter **\u003e\n  @type sanitizer\n  hash_salt mysalt\n  \u003crule\u003e\n    keys user.ssn\n    pattern_regex /^(?!(000|666|9))\\d{3}-(?!00)\\d{2}-(?!0000)\\d{4}$/\n    pattern_regex_prefix SSN\n  \u003c/rule\u003e\n  \u003crule\u003e\n    keys user.phone\n    pattern_regex /^\\d{3}-?\\d{3}-?\\d{4}$/\n    pattern_regex_prefix Phone\n  \u003c/rule\u003e\n\u003c/filter\u003e\n```\n**Input sample**\n```\n {\n     \"user\" : {\n         \"ssn\" : \"123-45-6789\"\n         \"phone\" : \"123-456-7890\"\n     }\n }\n```\n**Output sample**\n```\n {\n     \"user\" : {\n         \"ssn\" : \"SSN_f6b6430343a9a749e12db8a112ca74e9\"\n         \"phone\" : \"Phone_0a25187902a0cf755627397eb085d736\"\n     }\n }\n```\nFrom v0.1.2, \"regex_capture_group\" option is available. With \"regex_capture_group\" option, it is possible to mask specific part of original messages. \n\n**Configuration sample**\n```\n\u003crule\u003e\n  keys user.email\n  pattern_regex /(?\u003cuser\u003e\\w+)\\@\\w+.\\w+/\n  regex_capture_group \"user\"\n  pattern_regex_prefix \"USER\"\n\u003c/rule\u003e\n```\n**Input sample**\n```\n {\n     \"user\" : {\n         \"email\" : \"user1@demo.com\"\n     }\n }\n```\n**Output sample**\n```\n {\n     \"user\" : {\n         \"email\" : \"USER_321865df6f0ce6bdf3ea16f74623534a@demo.com\"\n     }\n }\n```\n\n### Tips : Debug how sanitizer works\nWhen you design custom rules in a configuration file, you might need information about how Sanitizer masks original values into hash values for debugging purposes. You can check that information if you run td-agent/Fluentd with debug option enabled. The debug information is shown in the log file of td-agent/Fluentd like the following log message sample.\n\n**Log message sample**\n```\nYYYY-MM-DD Time fluent.debug: {\"message\":\"[pattern_regex] sanitize '123-45-6789' to 'SSN_f6b6430343a9a749e12db8a112ca74e9'\"}\nYYYY-MM-DD Time fluent.debug: {\"message\":\"[pattern_regex] sanitize '123-456-7890' to 'Phone_0a25187902a0cf755627397eb085d736'\"}\n```\n## Contribute\nContribution to fluent-plugin-sanitizer is always welcomed.\n\n\n## Copyright\n* Copyright(c) 2021- TK Kubota\n* License\n  * Apache License, Version 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-sanitizer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluent%2Ffluent-plugin-sanitizer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluent%2Ffluent-plugin-sanitizer/lists"}