{"id":21082738,"url":"https://github.com/renproject/pack","last_synced_at":"2025-05-16T09:32:10.602Z","repository":{"id":46502637,"uuid":"279712780","full_name":"renproject/pack","owner":"renproject","description":"An application binary interface for well-typed message passing over networks and disks","archived":false,"fork":false,"pushed_at":"2022-07-21T02:54:22.000Z","size":6303,"stargazers_count":30,"open_issues_count":1,"forks_count":7,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-06-18T18:47:40.417Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/renproject.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-07-14T23:16:03.000Z","updated_at":"2024-01-31T23:09:26.000Z","dependencies_parsed_at":"2022-08-26T05:40:34.585Z","dependency_job_id":null,"html_url":"https://github.com/renproject/pack","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renproject%2Fpack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renproject%2Fpack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renproject%2Fpack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/renproject%2Fpack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/renproject","download_url":"https://codeload.github.com/renproject/pack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225419562,"owners_count":17471434,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-19T20:15:11.173Z","updated_at":"2024-11-19T20:15:11.854Z","avatar_url":"https://github.com/renproject.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `📦 pack`\n\n[![GitHub](https://github.com/renproject/pack/workflows/test/badge.svg)](https://github.com/renproject/pack/workflows/test/badge.svg)\n[![Coverage](https://coveralls.io/repos/github/renproject/pack/badge.svg?branch=master)](https://coveralls.io/github/renproject/pack?branch=master)\n[![Report](https://goreportcard.com/badge/github.com/renproject/pack)](https://goreportcard.com/badge/github.com/renproject/pack)\n\n[Documentation](https://godoc.org/github.com/renproject/pack)\n\nIn distributed systems, message passing often requires marshaling/unmarshaling messages in order to send them over the network. This is also useful for writing messages to logs, for persistence and debugging. The `📦 pack` library defines a generic application binary interface for serialising values without losing their type information. It supports:\n\n- [x] `Bool`,\n- [x] `U8`, `U16`, `U32`, `U16`, `U32`, `U64`, `U128`, `U256`,\n- [x] `String`, `Bytes`, `Bytes32`, `Bytes65`,\n- [x] `Struct`,\n- [x] `List`, and\n- [x] custom types.\n\n## Values\n\nThe primary interface for working with `📦 pack` is the `Value` interface. All values implement this interface, allowing them to be marshaling to binary and JSON, and to expose their type information.\n\n```go\nimport (\n    \"fmt\"\n\n    \"github.com/renproject/pack\"\n)\n\nfunc main() {\n    point := pack.NewStruct(\n        \"x\", pack.NewU64(42),\n        \"y\", pack.NewU64(100),\n    )\n    fmt.Printf(\"type: %v\", point.Type())\n}\n```\n\n## Types\n\nBy default, values do not marshal their types. This means that values cannot be unmarshalled, because there it is not always possible to know what type you are looking at without additional context. Consider the following examples:\n\n```go\nimport (\n    \"fmt\"\n\n    \"github.com/renproject/pack\"\n    \"github.com/renproject/surge\"\n)\n\nfunc main() {\n    x := pack.NewU64(1)\n    xData, _ := surge.ToBinary(x)\n\n    y := pack.NewString(\"1\")\n    yData, _ := surge.ToBinary(x)\n\n    fmt.Printf(\"x: %v\", pack.Bytes(xData))\n    fmt.Printf(\"y: %v\", pack.Bytes(yData))\n}\n```\n\nLooking only at the binary representation of the values, it is impossible to distingish between the types of these values when unmarshalling. This presents a common problem in distributed systems: how do my services tell each other about the type context? Well, with `📦 pack` we use the `Typed` value:\n\n```go\nimport (\n    \"encoding/json\"\n    \"fmt\"\n\n    \"github.com/renproject/pack\"\n)\n\nfunc main() {\n    typed := pack.NewTyped(\n        \"x\", pack.NewU64(1),\n        \"y\", pack.NewString(\"1\"),\n    )\n    fmt.Printf(\"type: %v\", typed.Type())\n\n    typedData, _ := json.MarshalIndent(typed, \"\", \"  \")\n    fmt.Printf(\"json: %v\", string(typedData))\n}\n```\n\nNow, we can see that the type information of our value has also been marshalled. In the case of JSON, the type information favours being verbose, so that it is easily debuggable by humans. However, the binary representation is much more compact. In practice, most services in distributed systems should use binary marshalling, unless they are in debug mode (binary marshalling is not only more compact, but it is also faster to marshal).\n\n## Kinds\n\nTypes are not always simple. In the case of integers, there is minimal information that we need to know: what kind of integer is it? The only answers are `U8`, `U16`, `U32`, `U64`, `U128`, and `U256`. However, structs and lists are more complex data types and the same question has an infinite possible answers. This is where _kinds_ are useful. The kind of a value can be thought of as the \"type of the type\". We can understand this better with a few examples:\n\n```go\nimport (\n    \"fmt\"\n\n    \"github.com/renproject/pack\"\n)\n\nfunc main() {\n    x := pack.NewStruct(\"foo\", pack.NewString(\"bar\"))\n    y := pack.NewStruct(\"bar\", pack.NewBool(true))\n\n    fmt.Printf(\"x type: %v\", x.Type())\n    fmt.Printf(\"y type: %v\", y.Type())\n\n    fmt.Printf(\"x kind: %v\", x.Type().Kind())\n    fmt.Printf(\"y kind: %v\", y.Type().Kind())\n}\n```\n\nWe can see from this example that, although `x` and `y` have different _types_, they both have the same _kind_; they are both structs. It turns out that the existence of kinds is necessary when marshaling/unmarshaling type information, but you should very rarely need to explicitly use kinds.\n\n## Custom Types\n\nIt is often convenient to use strongly-typed values at the language level (e.g. define/use custom Go structs). Using `📦 pack`, we can `Encode` and `Decode` to/from custom structs:\n\n```go\nimport (\n    \"fmt\"\n\n    \"github.com/renproject/pack\"\n)\n\ntype Foo struct {\n    X pack.U64 `json:\"x\"`\n    Y pack.U64 `json:\"y\"`\n}\n\nfunc main() {\n    foo := Foo {\n        X: pack.NewU64(1),\n        Y: pack.NewU64(2),\n    }\n    bar := pack.NewStruct(\n        \"x\", pack.NewU64(3),\n        \"y\", pack.NewU64(4),\n    )\n    \n    packed, err := pack.Encode(foo)\n    if err != nil {\n        panic(err)\n    }\n\n    fmt.Printf(\"foo type: %v\", packed.Type())\n    fmt.Printf(\"bar type: %v\", bar.Type())\n    \n    if err := pack.Decode(\u0026foo, bar); err != nil {\n        panic(err)\n    }\n    \n    fmt.Printf(\"foo.X: %v\", foo.X)\n    fmt.Printf(\"foo.Y: %v\", foo.Y)\n}\n```\n\n## Contribution\n\nBuilt with ❤ by Ren.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenproject%2Fpack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frenproject%2Fpack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frenproject%2Fpack/lists"}