{"id":20855974,"url":"https://github.com/solodynamo/custom-log-marshaler","last_synced_at":"2025-05-12T06:31:25.557Z","repository":{"id":65917301,"uuid":"601426417","full_name":"solodynamo/custom-log-marshaler","owner":"solodynamo","description":"Attempt to R.I.P PII or unnecessary info in logs and reduce log ingestion costs in the process.","archived":false,"fork":false,"pushed_at":"2023-05-09T02:38:55.000Z","size":37,"stargazers_count":14,"open_issues_count":2,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-01T03:37:05.694Z","etag":null,"topics":["cost-optimization","golang","logging","pii","piidata","zap","zerolog"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/solodynamo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-14T03:10:40.000Z","updated_at":"2024-10-03T05:16:58.000Z","dependencies_parsed_at":"2024-06-20T16:44:29.113Z","dependency_job_id":"3de8c145-a168-4b0c-a71c-f5f485eeaeaf","html_url":"https://github.com/solodynamo/custom-log-marshaler","commit_stats":null,"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solodynamo%2Fcustom-log-marshaler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solodynamo%2Fcustom-log-marshaler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solodynamo%2Fcustom-log-marshaler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/solodynamo%2Fcustom-log-marshaler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/solodynamo","download_url":"https://codeload.github.com/solodynamo/custom-log-marshaler/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253687529,"owners_count":21947692,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cost-optimization","golang","logging","pii","piidata","zap","zerolog"],"created_at":"2024-11-18T04:27:48.361Z","updated_at":"2025-05-12T06:31:25.302Z","avatar_url":"https://github.com/solodynamo.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🕵️ Custom Log Marshaler\n\n\"Don't log any data that is unnecessary or should not be logged in the first place.\" - common sense\n\nWe can tag PII struct fields as \"notloggable\" and this generator will output custom marshal functions for those Golang structs which will prevent sending those fields to stdout!\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://goreportcard.com/report/github.com/solodynamo/custom-log-marshaler\"\u003e\n    \u003cimg src=\"https://goreportcard.com/badge/github.com/solodynamo/custom-log-marshaler\" /\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://github.com/solodynamo/custom-log-marshaler/releases\"\u003e\n    \u003cimg src=\"https://img.shields.io/github/release/solodynamo/custom-log-marshaler.svg\" /\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## How?\n\nIn most Go logging libraries, there is a way to override the default marshaling function used to \"log\" to stdout. This package generates that custom function with some superpowers like excluding fields (for PII, not required stuff).\n\nIf you send less data, ingestion bandwidth is used less so this also leads to lesser costs in the long run.\n\nExample:\n```go\ntype User struct {\n\tName    string `json:\"name\"`\n\tEmail   string `notloggable`\n\tAddress string `json:\"address\", notloggable`\n}\n\ntype UserDetailsResponse struct {\n\tUser\n\tRequestID string   `json:\"rid\"`\n\tFromCache bool     `json:\"fromCache\"`\n\tMetadata  []string `json:\"md\"`\n}\n// MarshalLogObject ...\nfunc (l User) MarshalLogObject(enc zapcore.ObjectEncoder) error {\n\tenc.AddString(\"name\", l.Name) // not logging things which shouldn't be logged.\n\treturn nil\n}\n\n// MarshalLogObject ...\nfunc (l UserDetailsResponse) MarshalLogObject(enc zapcore.ObjectEncoder) error {\n\tenc.AddObject(\"user\", l.User)\n\tenc.AddString(\"request_id\", l.RequestID)\n\tenc.AddBool(\"from_cache\", l.FromCache)\n\tenc.AddArray(\"metadata\", l.Metadata)\n\treturn nil\n}\n```\n\nSee above example in action on [Go Playground](https://go.dev/play/p/cv_u168fm0e?v=goprev). \n\n# Why? \n\nCouldn't find something like this.\n\n# Installation\n\nMacOS\n```bash\nbrew tap solodynamo/homebrew-tap\nbrew install custom-log-marshaler\n```\n\nOthers: \n```bash\ngo install github.com/solodynamo/custom-log-marshaler\n```\n\n# Usage\n\nZerolog\n\n```bash\ncustom-log-marshaler -f \"path to go file\" -lib zerolog\n```\n\nUber Zap(by default)\n\n```bash\ncustom-log-marshaler -f \"path to go file\"\n\n```\n\nFor bulk operation on multiple go files: \n\n```bash\ncurl -sSL https://gist.githubusercontent.com/solodynamo/b23de7cc6576179292871efc9b37e1f1/raw/apply-clm-go.sh | bash -s -- \"path1/to/ignore\" \"path2/to/ignore\"\n\n```\nNote: Above bulk script handles the installation too but only for MacOS.\n\n# How to exclude PII? \n```go\ntype User struct {\n\tEmail   string `notloggable`\n}\n```\nWhen custom struct tag `notloggable` is used for a field, that field is excluded from final logging irrespective of the lib. \n\n# Contributions\nPlease do!\n\n# Just a joke\n\u003cimg width=\"789\" alt=\"chatgptjoke\" src=\"https://user-images.githubusercontent.com/17698714/220406972-0a42c233-fe71-4f58-b337-f10deb4f171c.png\"\u003e","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolodynamo%2Fcustom-log-marshaler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsolodynamo%2Fcustom-log-marshaler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsolodynamo%2Fcustom-log-marshaler/lists"}