{"id":51917413,"url":"https://github.com/longbridge/json-transformer","last_synced_at":"2026-07-27T13:04:07.505Z","repository":{"id":351950328,"uuid":"1213173177","full_name":"longbridge/json-transformer","owner":"longbridge","description":"One-pass streaming JSON field renaming and value transformation for Go","archived":false,"fork":false,"pushed_at":"2026-04-17T06:24:12.000Z","size":20,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-17T08:28:13.458Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/longbridge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-17T05:43:27.000Z","updated_at":"2026-04-17T06:24:15.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/longbridge/json-transformer","commit_stats":null,"previous_names":["longbridge/json-transformer"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/longbridge/json-transformer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longbridge%2Fjson-transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longbridge%2Fjson-transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longbridge%2Fjson-transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longbridge%2Fjson-transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/longbridge","download_url":"https://codeload.github.com/longbridge/json-transformer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/longbridge%2Fjson-transformer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35951500,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-27T02:00:06.776Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-27T13:04:06.825Z","updated_at":"2026-07-27T13:04:07.488Z","avatar_url":"https://github.com/longbridge.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# json-transformer\n\n## Why\n\nGo's `encoding/json` gives you no control over the JSON output once your struct is defined. Real-world systems frequently need to change the JSON shape without touching the data model:\n\n- A service receives `camelCase` JSON from a frontend but must forward `snake_case` to a downstream API.\n- An API gateway needs to mask sensitive fields (passwords, tokens) before logging.\n- A data pipeline reads NDJSON, renames fields to match a new schema, and streams the result to another sink.\n\nThe standard workarounds all have significant costs:\n\n| Workaround | Problem |\n|---|---|\n| Second struct with different tags | Doubles type definitions; breaks with dynamic schemas |\n| `json.Unmarshal` → rename → `json.Marshal` | Two full encode/decode passes; entire document loaded into memory |\n| `map[string]any` manipulation | Verbose, error-prone, loses type information |\n| Custom `MarshalJSON` per type | Couples serialisation logic to the domain model |\n\n**json-transformer** solves this in one pass: rename keys and transform values as the JSON flows through, without loading the full document into memory.\n\n## Design\n\n**One pass, no round-trips.** When the input is a Go value (`map`, `struct`, `slice`), the library walks it directly and writes JSON with no intermediate marshal step. When the input is raw JSON (`[]byte`, `string`, `io.Reader`), a token-based decoder drives the output writer in lockstep — the document is never fully decoded into memory.\n\n**Zero config, zero cost.** A `Transformer` with no options short-circuits to a direct pass-through. You pay only for what you configure.\n\n**Concurrency-safe.** All per-call state is local to each invocation. One `Transformer` can be shared freely across goroutines.\n\n**Follows `encoding/json` conventions.** Struct tags (`json:\"name,omitempty\"`, `json:\"-\"`, anonymous embedding) are respected. Map key order is non-deterministic, consistent with the standard library. Large numbers are preserved exactly via `json.Number` with no float64 precision loss.\n\n## Installation\n\n```bash\ngo get github.com/longbridge/json-transformer\n```\n\n## Quick start\n\n```go\nimport jsontransform \"github.com/longbridge/json-transformer\"\n\nt := jsontransform.New(\n    jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()),\n)\n\nout, err := t.TransformBytes(map[string]any{\"userName\": \"Alice\", \"userAge\": 30})\n// out: {\"user_name\":\"Alice\",\"user_age\":30}\n```\n\n## API\n\n### Creating a transformer\n\n```go\nt := jsontransform.New(opts ...Option) *Transformer\n```\n\n| Option | Description |\n|---|---|\n| `WithRenameFunc(fn RenameFunc)` | Called for every object key. Return `nil` to keep the original name, or `*string` with the new name. |\n| `WithValueTransformer(fn func(string) ValueTransformFunc)` | Called once per unique field name. Return a `ValueTransformFunc` to transform that field's value, or `nil` to leave it unchanged. |\n\n### Choosing a method\n\n| Method | Use when |\n|---|---|\n| `TransformBytes(src any) ([]byte, error)` | You need the result as `[]byte`. The most common case. |\n| `Transform(src any, dst io.Writer) error` | You already have an `io.Writer` to write into (HTTP response, file, network connection). Also the right choice when `src` is an `io.Reader`. |\n| `TransformStream(r io.Reader, w io.Writer) error` | Your input contains **multiple JSON values** in sequence (NDJSON). Each value is transformed and written on its own line. Use `Transform` for single-value input. |\n\n```go\nout, err := t.TransformBytes(src)       // → []byte\nerr      := t.Transform(src, w)         // → io.Writer\nerr      := t.TransformStream(r, w)     // NDJSON: multiple values\n```\n\n### Input dispatch\n\n`Transform` and `TransformBytes` accept any of the following as `src`:\n\n| Type | Behaviour |\n|---|---|\n| `map[string]any` | Fast path — traverses the map directly, no marshal/unmarshal |\n| `struct` / `*struct` | Fast path — reflection with cached field metadata |\n| `slice` / `array` | Fast path |\n| `[]byte` / `string` / `io.Reader` | Streaming path — parses JSON tokens without loading the full document |\n| Primitives (`int`, `bool`, …) | Encoded directly with `encoding/json` |\n\n## Built-in rename functions\n\n```go\njsontransform.SnakeCaseRename()                          // \"UserName\"  → \"user_name\"\njsontransform.CamelCaseRename()                          // \"user_name\" → \"userName\"\njsontransform.PascalCaseRename()                         // \"user_name\" → \"UserName\"\njsontransform.KebabCaseRename()                          // \"UserName\"  → \"user-name\"\njsontransform.MapRename(map[string]string{\"id\": \"ID\"})   // exact lookup; returns nil if not found\n```\n\n`MapRename` returns `nil` for unknown keys, so it composes naturally with a fallback:\n\n```go\nexact    := jsontransform.MapRename(map[string]string{\"id\": \"ID\"})\nfallback := jsontransform.SnakeCaseRename()\n\nt := jsontransform.New(\n    jsontransform.WithRenameFunc(func(name string) *string {\n        if s := exact(name); s != nil {\n            return s\n        }\n        return fallback(name)\n    }),\n)\n```\n\n## Examples\n\n### Rename fields on a struct\n\n```go\ntype User struct {\n    UserName string `json:\"userName\"`\n    UserAge  int    `json:\"userAge\"`\n}\n\nt := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()))\nout, _ := t.TransformBytes(User{UserName: \"Alice\", UserAge: 30})\n// {\"user_name\":\"Alice\",\"user_age\":30}\n```\n\n### Rename a key and mask its value\n\n`WithValueTransformer` receives the **original** field name. Return a transform function for fields you want to handle, or `nil` to leave the value unchanged. Renaming and value transformation are independent and can be combined freely.\n\n```go\nt := jsontransform.New(\n    jsontransform.WithRenameFunc(func(name string) *string {\n        if name == \"pwd\" {\n            s := \"password\"\n            return \u0026s\n        }\n        return nil\n    }),\n    jsontransform.WithValueTransformer(func(name string) jsontransform.ValueTransformFunc {\n        if name == \"pwd\" {\n            return func(v any) any { return \"***\" }\n        }\n        return nil\n    }),\n)\n\nout, _ := t.TransformBytes(map[string]any{\"user\": \"alice\", \"pwd\": \"secret\"})\n// {\"user\":\"alice\",\"password\":\"***\"}\n```\n\n### Transform values by pattern\n\nBecause `WithValueTransformer` gives you the field name, you can match by any pattern rather than a fixed string:\n\n```go\nt := jsontransform.New(\n    jsontransform.WithValueTransformer(func(name string) jsontransform.ValueTransformFunc {\n        if strings.HasSuffix(name, \"_at\") {\n            return func(v any) any {\n                // reformat Unix timestamp → RFC3339\n                if n, ok := v.(json.Number); ok {\n                    ts, _ := n.Int64()\n                    return time.Unix(ts, 0).UTC().Format(time.RFC3339)\n                }\n                return v\n            }\n        }\n        return nil\n    }),\n)\n\nout, _ := t.TransformBytes(map[string]any{\"created_at\": 1700000000, \"name\": \"Alice\"})\n// {\"created_at\":\"2023-11-14T22:13:20Z\",\"name\":\"Alice\"}\n```\n\n### Stream a large JSON file\n\n```go\nt := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.CamelCaseRename()))\n\nf, _ := os.Open(\"large.json\")\ndefer f.Close()\nt.Transform(f, os.Stdout)\n```\n\n### Process NDJSON\n\n```go\nt := jsontransform.New(jsontransform.WithRenameFunc(jsontransform.SnakeCaseRename()))\n\nr := strings.NewReader(`{\"userId\":1}` + \"\\n\" + `{\"userId\":2}`)\nt.TransformStream(r, os.Stdout)\n// {\"user_id\":1}\n// {\"user_id\":2}\n```\n\n## Notes\n\n- **Field name matching** (both `WithValueTransformer` and `RenameFunc`) always uses the original field name, before any renaming is applied. The lookup function passed to `WithValueTransformer` is called at most once per unique field name; results are cached for the lifetime of the `Transformer`.\n- **Value transformer input** when processing JSON text (`[]byte`/`string`/`io.Reader`): values arrive as `string`, `json.Number`, `bool`, `nil`, `map[string]any`, or `[]any`. When processing Go values directly, the original Go type is passed.\n- **Value transformer output** is encoded as-is. It is not subject to further renaming or transformation.\n- **Map key order** is non-deterministic, consistent with `encoding/json`.\n- **Large numbers** in JSON text are preserved exactly via `json.Number`; there is no float64 precision loss.\n\n## Benchmarks\n\nMeasured on Windows amd64, Intel Core i9-12900K, Go 1.24.\n\n```\nBenchmarkTransformMap          ~487 ns/op     1 alloc/op\nBenchmarkTransformStruct       ~340 ns/op     2 allocs/op\nBenchmarkTransformParallel     ~199 ns/op     1 alloc/op   (24 goroutines)\nBenchmarkTransformBytes        ~899 ns/op    13 allocs/op  ([]byte input, fastjson path)\nBenchmarkTransformNoOp          ~36 ns/op     1 alloc/op   (pass-through baseline, Writer)\nBenchmarkTransformLargeJSON    ~975 ms/op   ~108 MB/s      (100 MB JSON array, []byte input)\n```\n\nThe `[]byte`/`string`/`io.Reader` path is driven by [fastjson](https://github.com/valyala/fastjson)'s arena parser. The fast path (in-memory Go values) has no token-allocation overhead at all.\n\nRun on your own machine:\n\n```bash\ngo test -bench=. -benchmem ./...\n```\n\n## License\n\nMIT — see [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flongbridge%2Fjson-transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flongbridge%2Fjson-transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flongbridge%2Fjson-transformer/lists"}