{"id":51317691,"url":"https://github.com/tochemey/goserde","last_synced_at":"2026-07-05T13:00:36.136Z","repository":{"id":368163652,"uuid":"1283268959","full_name":"Tochemey/goserde","owner":"Tochemey","description":"[Go] A code-generated, zero-reflection binary serializer","archived":false,"fork":false,"pushed_at":"2026-07-03T09:35:05.000Z","size":181,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-07-04T12:21:50.613Z","etag":null,"topics":["codegen","go","high-performance","serialization","zero-allocation","zero-dependencies"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Tochemey.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-28T18:28:24.000Z","updated_at":"2026-07-03T09:34:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Tochemey/goserde","commit_stats":null,"previous_names":["tochemey/goserde"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/Tochemey/goserde","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tochemey%2Fgoserde","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tochemey%2Fgoserde/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tochemey%2Fgoserde/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tochemey%2Fgoserde/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Tochemey","download_url":"https://codeload.github.com/Tochemey/goserde/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Tochemey%2Fgoserde/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35154748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-05T02:00:06.290Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["codegen","go","high-performance","serialization","zero-allocation","zero-dependencies"],"created_at":"2026-07-01T09:02:50.701Z","updated_at":"2026-07-05T13:00:36.101Z","avatar_url":"https://github.com/Tochemey.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# goSerde\n\n[![GitHub Actions Workflow Status](https://img.shields.io/github/actions/workflow/status/tochemey/goserde/ci.yml)](https://github.com/tochemey/goserde/actions/workflows/ci.yml)\n[![codecov](https://img.shields.io/codecov/c/github/tochemey/goserde?branch=main)](https://codecov.io/gh/tochemey/goserde)\n[![Go Reference](https://pkg.go.dev/badge/github.com/tochemey/goserde.svg)](https://pkg.go.dev/github.com/tochemey/goserde)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\nA code-generated, zero-reflection binary serializer for Go, built for maximum\nencode/decode throughput when you own both ends of the wire.\n\ngoserde generates type-specific `Size`/`Marshal`/`Unmarshal` methods for your\nstructs at build time. There is no reflection on the hot path, no schema\nlanguage, and no runtime type registry: just straight-line Go over a single\nrunning offset, with inlinable primitives and zero-copy decoding.\n\n```\nMarshal    14 ns/op    0 allocs     (28× faster than encoding/json)\nUnmarshal  28 ns/op    1 alloc      (81× faster than encoding/json)\n```\n\n## Is goSerde for you?\n\n**Use it when** you control both the encoder and the decoder and want the\nsmallest, fastest possible binary representation: RPC between your own services,\non-disk caches, IPC, or embedded pipelines. The default **fast mode** assumes\ntrusted input on the same architecture; an opt-in **safe mode** adds\nbounds-checked decoding, copies of decoded data, and a portable little-endian\nformat for untrusted or cross-machine input (see [Modes](#modes)).\n\n**Look elsewhere when** you need schema evolution, a self-describing format, or\nforward/backward compatibility with peers on different versions. goserde's schema\nlives entirely in the generated code, with no field tags or type markers on the\nwire, so the format does not evolve on its own. For that, reach for a\nschema-evolving format such as Protobuf, Cap'n Proto, or FlatBuffers.\n\n## Install\n\n```bash\n# The codec support library (imported by generated code)\ngo get github.com/tochemey/goserde\n\n# The code generator\ngo install github.com/tochemey/goserde/cmd/goserdegen@latest\n```\n\nThe library has **zero external dependencies** and builds offline.\n\n## Quick start\n\nAnnotate a struct with the `//goserde:generate` directive:\n\n```go\npackage model\n\n//go:generate goserdegen -dir . -out goserde_gen.go\n\n//goserde:generate\ntype User struct {\n\tID   uint64\n\tName string\n\tTags []int32\n}\n```\n\nGenerate the codec:\n\n```bash\ngo generate ./...\n```\n\nEncode and decode with the convenience helpers:\n\n```go\nimport \"github.com/tochemey/goserde/codec\"\n\nu := \u0026User{ID: 1, Name: \"ada\", Tags: []int32{1, 2, 3}}\n\n// Encode into a fresh, exactly sized buffer.\ndata := codec.Bytes(u)\n\n// Decode.\nvar out User\nif err := codec.From(\u0026out, data); err != nil {\n\tlog.Fatal(err)\n}\n```\n\nOn the hot path, drive the generated methods directly and reuse buffers to stay\nallocation-free:\n\n```go\nbuf = codec.Into(u, buf) // reuses buf when it has capacity, else allocates\n\nvar out User\nout.Unmarshal(data)        // returns (bytesRead int, err error)\n```\n\n## Supported types\n\n| Category   | Types                                                                        |\n|------------|------------------------------------------------------------------------------|\n| Integers   | `int`, `int8/16/32/64`, `uint`, `uint8/16/32/64`                             |\n| Floats     | `float32`, `float64`                                                         |\n| Scalars    | `bool`, `string`, `[]byte`                                                   |\n| Composites | slices `[]T`, fixed arrays `[N]T`, maps `map[K]V`, pointers `*T`             |\n| Structs    | nested structs (each annotated with `//goserde:generate`)                    |\n| Time       | `time.Time` (as int64 Unix-nanoseconds, UTC)                                 |\n| Unions     | interface fields via `//goserde:union` (see [Tagged unions](#tagged-unions)) |\n\nFields tagged `` `goserde:\"-\"` `` are excluded from the wire and decode back to\ntheir zero value, useful for fields of types goserde cannot serialize\n(channels, funcs).\n\n```go\n//goserde:generate\ntype Session struct {\n\tID    uint64\n\tToken string\n\tconn  net.Conn `goserde:\"-\"` // excluded\n}\n```\n\n### Tagged unions\n\nDeclare a polymorphic field by listing its concrete members above an interface.\nEach member must itself be a `//goserde:generate` struct whose pointer implements\nthe interface:\n\n```go\n//goserde:generate\ntype Circle struct{ R float64 }\n\nfunc (*Circle) isShape() {}\n\n//goserde:generate\ntype Square struct{ Side float64 }\n\nfunc (*Square) isShape() {}\n\n//goserde:union Circle Square\ntype Shape interface{ isShape() }\n\n//goserde:generate\ntype Drawing struct {\n\tName  string\n\tShape Shape   // holds *Circle, *Square, or nil\n}\n```\n\nOn the wire a union is a varint tag (`0` = nil, `1` = first member, …) followed\nby the member's payload. **Member order is the wire contract**: only ever\nappend new members, never reorder. Decoding an unknown tag returns\n`codec.ErrUnknownUnionTag` rather than panicking.\n\n## Modes\n\nA package is generated in exactly one mode; the method signatures are identical,\nso callers don't change.\n\n| Mode               | Flag    | Unmarshal on bad input         | Decoded `string`/`[]byte` | Portable            |\n|--------------------|---------|--------------------------------|---------------------------|---------------------|\n| **Fast** (default) | none    | may panic (trusted bytes)      | zero-copy (aliases input) | no (native memory)  |\n| **Safe**           | `-safe` | returns `codec.ErrShortBuffer` | copied (owns its bytes)   | yes (little-endian) |\n\nFast mode is for trusted bytes on identical architectures: it uses `unsafe`\nzero-copy decoding and a single `memmove` for all-fixed-width structs. Safe mode\nbounds-checks every read, copies length-prefixed payloads, and encodes\nfield-by-field in little-endian, portable across architectures and Go versions,\nat a modest cost.\n\n**Decoding untrusted input? Use safe mode.** Fast mode trusts its bytes, and\nthat has two consequences. First, malformed input may panic or read out of\nbounds, so never decode bytes you did not produce yourself (network payloads,\nfiles written by other processes, user input) in fast mode. Second, a decoded\n`string` or `[]byte` aliases the buffer passed to `Unmarshal` rather than\ncopying, so that buffer must outlive the decoded value and must not be mutated\nor returned to a pool while the value is in use. Safe mode removes both hazards:\nit returns `codec.ErrShortBuffer` on truncated input and copies decoded data so\nthe result owns its bytes.\n\n```bash\ngoserdegen -dir . -out goserde_gen.go        # fast (default)\ngoserdegen -dir . -out goserde_gen.go -safe  # safe / portable\n```\n\n## Performance\n\nAll numbers below are from the same machine (Go 1.26, darwin/arm64, Apple M1).\nReproduce with `make bench` (vs the standard library), `make bench-shapes`\n(across shapes), and `make compare` (vs mus and benc).\n\n### vs the standard library\n\nFlat `Record` struct (`uint64`, `float64`, `bool`, `string`, `[]uint32`,\n`[]byte`):\n\n| Operation | goserde            | encoding/json      | encoding/gob¹ |\n|-----------|--------------------|--------------------|---------------|\n| Marshal   | **14 ns**, 0 alloc | 399 ns, 1 alloc    | 191 ns        |\n| Unmarshal | **28 ns**, 1 alloc | 2258 ns, 12 allocs | n/a           |\n\n¹ gob marshal uses a reused encoder. goserde is ~28× faster than JSON on\nmarshal and ~81× on unmarshal, with a fraction of the allocations.\n\n### vs other fast serializers\n\nSame `Record` data, 3 runs each, with [mus](https://github.com/mus-format/mus-go)\nv0.10.2 and [benc](https://github.com/deneonet/benc) v1.1.8:\n\n| Serializer   | Marshal     | Unmarshal   | Payload | Decode allocs |\n|--------------|-------------|-------------|---------|---------------|\n| **goserde**  | **13.3 ns** | **28.0 ns** | 139 B   | **1**         |\n| benc         | 25.3 ns     | 45.1 ns     | 143 B   | 1             |\n| mus (raw)    | 34.3 ns     | 68.6 ns     | 139 B   | 2             |\n| mus (varint) | 34.2 ns     | 73.0 ns     | 109 B   | 2             |\n\ngoserde is ~1.9× / ~1.6× faster than benc (marshal/unmarshal) and ~2.6× faster\nthan mus on both. The main reason is **no interface dispatch in the generated\ncode**: running mus in fixed-width \"raw\" mode produces goserde's exact 139 B\nformat yet barely changes its speed, so the win is the straight-line code, not\nthe format. mus and benc are fuller-featured libraries (schema evolution,\nvalidation, versioning); goserde trades those for raw throughput.\n\n### Across struct shapes\n\ngoserde's lead is shape-dependent. Marshal is zero-alloc on every shape:\n\n| Shape               | Marshal | Unmarshal | Decode allocs | Notes                          |\n|---------------------|---------|-----------|---------------|--------------------------------|\n| Fixed-width (4 fld) | 2.4 ns  | 2.4 ns    | 0             | single memmove (blittable)     |\n| Flat + string/bytes | 7.1 ns  | 6.3 ns    | 0             | zero-copy `string`/`[]byte`    |\n| String-heavy        | 19.6 ns | 33.0 ns   | 1             | `[]string` must allocate       |\n| Nested + pointers   | 16.9 ns | 36.1 ns   | 2             | pointer + slice-of-struct      |\n| Map-heavy           | 77.5 ns | 151.9 ns  | 4             | `make(map)` + per-entry insert |\n\n**Decode reuse:** decoding repeatedly into the *same* value reuses its slices\n(by capacity) and maps (via `clear`). The map-heavy shape above drops from\n**152 ns / 4 allocs to 67 ns / 0 allocs** when the destination is reused across\ncalls, the right pattern for a read loop.\n\nFixed and flat structs are where goserde dominates. Map-heavy decode is its\nweakest point on a fresh destination, where mus and benc are competitive.\n\n## Wire format\n\nFixed-width little-endian numerics; LEB128 varint length prefixes for\nstrings, slices, and maps; a 1-byte nil flag for pointers; an 8-byte\nUnix-nanosecond int64 for `time.Time`. All-fixed-width structs are encoded as a\nsingle memory copy in fast mode. There are no field names, tags, or type\nmarkers on the wire: the schema lives entirely in the generated code.\n\nThis format is **not** self-describing and, in fast mode, **not** portable across\narchitectures or Go versions. That is the deliberate trade for speed.\n\n## Compatibility\n\ngoserde is pre-1.0: the API and wire format may change between minor versions\nuntil 1.0.0. From 1.0.0 onward the following semantic-versioning promise applies.\n\n**Covered by SemVer (stable within a major version):**\n\n- The generated method set on your types: `Size() int`, `Marshal([]byte) int`,\n  and `Unmarshal([]byte) (int, error)`.\n- The full `codec` package: the convenience API (`Bytes`, `Into`, `From`, and\n  the `Marshaler` / `Unmarshaler` interfaces) and the low-level encoding\n  primitives (`PutU32`/`U32` and friends, `PutUvarint`/`Uvarint`/`UvarintSize`,\n  `Zig`/`Zag`, `B2S`/`S2B`, the float bit-casts) you can use to hand-write a\n  codec instead of running the generator. Generated and hand-written codecs call\n  the same primitives, so they share an identical wire format.\n- The generator directives `//goserde:generate` and `//goserde:union`, and the\n  `goserde:\"-\"` struct tag.\n\n**Wire-format stability:**\n\n- **Safe mode** (`-safe`) is the portable format: little-endian and\n  bounds-checked, stable across architectures and Go versions. Use it for data\n  at rest or exchanged between machines.\n- **Fast mode** (default) is not portable. It uses native byte order and encodes\n  all-fixed-width structs as a copy of their in-memory layout, so the bytes can\n  differ across architectures and across Go versions whose struct layout\n  differs. Treat fast-mode bytes as ephemeral and decode them with the same\n  goserde version, architecture, and Go toolchain that produced them.\n\nRegenerating codecs after upgrading goserde is always safe and recommended.\n\n## How it works\n\nThe generator (`cmd/goserdegen`) is **standard-library only**: it loads and\ntype-checks your package with `go/parser` and `go/types` (no `go/packages`, no\nnetwork), finds `//goserde:generate` structs, and emits a codec shaped exactly\nlike the hand-tuned reference in [`codec/record.go`](codec/record.go).\nGenerated code imports goserde's `codec` package, whose primitives (varint,\nzigzag, little-endian fixed-width R/W, zero-copy conversions, float bit-casts)\nare written to inline.\n\nThe import path of that `codec` package is derived automatically from the\nmodule that built `goserdegen`, so forking under a different module path \"just\nworks\" with no flags to keep in sync.\n\n## Development\n\n```bash\ngo build ./...        # build everything (zero dependencies)\ngo test ./...         # run the full test suite\ngo generate ./...     # regenerate codecs after editing annotated structs\nmake bench-shapes     # benchmark across struct shapes\nmake compare          # head-to-head vs mus and benc (needs network)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftochemey%2Fgoserde","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftochemey%2Fgoserde","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftochemey%2Fgoserde/lists"}