{"id":22830750,"url":"https://github.com/ndimensional/zig-ipld","last_synced_at":"2025-03-31T01:41:23.533Z","repository":{"id":264273804,"uuid":"892848643","full_name":"nDimensional/zig-ipld","owner":"nDimensional","description":null,"archived":false,"fork":false,"pushed_at":"2025-03-11T17:51:53.000Z","size":89,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-31T01:41:22.732Z","etag":null,"topics":["dag-cbor","dag-json","ipld","zig"],"latest_commit_sha":null,"homepage":"","language":"Zig","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nDimensional.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-22T22:34:48.000Z","updated_at":"2025-03-11T17:51:39.000Z","dependencies_parsed_at":"2025-03-11T18:41:34.132Z","dependency_job_id":null,"html_url":"https://github.com/nDimensional/zig-ipld","commit_stats":null,"previous_names":["ndimensional/zig-ipld"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nDimensional%2Fzig-ipld","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nDimensional%2Fzig-ipld/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nDimensional%2Fzig-ipld/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nDimensional%2Fzig-ipld/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nDimensional","download_url":"https://codeload.github.com/nDimensional/zig-ipld/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246403892,"owners_count":20771526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dag-cbor","dag-json","ipld","zig"],"created_at":"2024-12-12T20:14:16.346Z","updated_at":"2025-03-31T01:41:23.504Z","avatar_url":"https://github.com/nDimensional.png","language":"Zig","funding_links":[],"categories":[],"sub_categories":[],"readme":"# zig-ipld\n\nZig implementation of the [IPLD](https://ipld.io/) data model, with dag-cbor and dag-json codecs.\n\nProvides two parallel APIs:\n\n- a \"dynamic\" heap-allocated `Value` type for manipulating arbitrary/heterogenous values\n- a \"static\" API for generating encoders and decoders for native Zig types at comptime\n\nPasses all tests in [ipld/codec-fixtures](https://github.com/ipld/codec-fixtures) except for `i64` integer overflow cases.\n\n## Table of Contents\n\n- [Install](#install)\n- [Usage](#usage)\n  - [Creating dynamic values](#creating-dynamic-values)\n  - [Encoding dynamic values](#encoding-dynamic-values)\n  - [Decoding dynamic values](#decoding-dynamic-values)\n  - [Static types](#static-types)\n  - [Strings and bytes](#strings-and-bytes)\n\n## Install\n\nAdd to `build.zig.zon`:\n\n```zig\n.{\n    // ...\n    .dependencies = .{\n        .ipld = .{\n            .url = \"https://github.com/nDimensional/zig-ipld/archive/${COMMIT}.tar.gz\",\n            // .hash = \"...\",\n        },\n    }\n}\n```\n\nThen in `build.zig`:\n\n```zig\npub fn build(b: *std.Build) !void {\n    const ipld_dep = b.dependency(\"ipld\", .{});\n\n    const ipld = ipld_dep.module(\"ipld\");\n    const json = ipld_dep.module(\"dag-json\");\n    const cbor = ipld_dep.module(\"dag-cbor\");\n\n    // All of the modules from zig-multiformats are also re-exported\n    // from zig-ipld, for convenience and to ensure consistent versions.\n    const varint = ipld_dep.module(\"varint\");\n    const multibase = ipld_dep.module(\"multibase\");\n    const multihash = ipld_dep.module(\"multihash\");\n    const cid = ipld_dep.module(\"cid\");\n\n    // add as imports to your exe or lib...\n}\n```\n\n## Usage\n\n### Creating dynamic Values\n\n```zig\nconst ipld = @import(\"ipld\");\nconst Kind = ipld.Kind;\nconst Value = ipld.Value;\n\n// ipld.Value is a union over ipld.Kind: enum {\n//   null, boolean, integer, float, string, bytes, list, map, link\n// }\n\n// the kinds [null, boolean, integer, float] have static values.\n// the kinds [string, bytes, list, map, link] are pointers to\n// allocated ref-counted objects\n\nconst false_val = Value.False;\nconst true_val = Value.True;\nconst null_val = Value.Null;\nconst float_val = Value.float(15.901241);\n\n// integer values are all i64.\n// integer values outside this range are not supported.\nconst int_val = Value.integer(-8391042);\n\npub fn main() !void {\n    // this copies the contents of the argument,\n    // the object string_val owns its own data.\n    const string_val = try Value.createString(allocator, \"hello world\");\n    defer string_val.unref(); // unref() decrements the refcount\n\n    const list_val = try Value.createList(allocator, .{});\n\n    // appending an existing value here will increment string_val's refcount,\n    // keeping it alive for as long as list_val lives.\n    try list_val.list.append(string_val);\n\n    // for convenience, you can also pass a tuple of \"initial values\" to createList.\n    // this *will not* increment the refcount of the initial values, which is\n    // useful for initializing deeply nested objects in a single expression.\n    const list_val2 = try Value.createList(allocator, .{\n        try Value.createList(allocator, .{\n            try Value.createList(allocator, .{\n                try Value.createList(allocator, .{}),\n            }),\n        }),\n    });\n    defer list_val2.unref();\n    // this created [[[[]]]], which will all be freed\n    // when `defer list_val2.unref()` is invoked\n\n    // maps are similar, with an initial struct argument that\n    // doesn't increment initial value's refcounts\n    const map_val = try Value.createMap(allocator, .{\n        .foo = try Value.createString(allocator, \"initial value 1\"),\n        .bar = try Value.createString(allocator, \"initial value 2\"),\n    });\n    defer map_val.unref();\n\n    // this increments string_val's refcount.\n    // map keys are copied with the map's allocator and managed by the map.\n    try map_val.map.set(\"baz\", string_val);\n}\n```\n\n### Encoding dynamic values\n\n`dag-cbor` and `dag-json` both export `Encoder` structs with identical APIs. Here's an example with a dag-json encoder:\n\n```zig\nconst std = @import(\"std\");\nconst ipld = @import(\"ipld\");\nconst Value = ipld.Value;\n\ntest \"encode a dynamic value\" {\n    const allocator = std.heap.c_allocator;\n    const example_value = try Value.createList(allocator, .{\n        try Value.createList(allocator, .{}),\n        try Value.createList(allocator, .{\n            Value.Null,\n            Value.integer(42),\n            Value.True,\n        }),\n    });\n\n    var encoder = json.Encoder.init(allocator, .{});\n    defer encoder.deinit();\n\n    const bytes = try encoder.encodeValue(allocator, example_value);\n    defer allocator.free(bytes);\n\n    try std.testing.expectEqualSlices(u8, \"[[],[null,42,true]]\", json_bytes);\n}\n```\n\nThe encoders must be initialized with an allocator, which is used for internal encoder state. Each `encodeValue` call takes its own allocator that it **only** uses for allocating the resulting `[]const u8` slice.\n\n### Decoding dynamic values\n\n`dag-cbor` and `dag-json` both export `Decoder` structs with identical APIs.\n\nHere's an example with a dag-json decoder:\n\n```zig\nconst std = @import(\"std\");\nconst ipld = @import(\"ipld\");\nconst Value = ipld.Value;\n\ntest \"decode a dynamic value\" {\n    const allocator = std.heap.c_allocator;\n\n    var decoder = json.Decoder.init(allocator, .{});\n    defer decoder.deinit();\n\n    const value = try decoder.decodeValue(allocator, \"[[],[null,42,true]]\");\n    defer value.unref();\n\n    const expected_value = try Value.createList(allocator, .{\n        try Value.createList(allocator, .{}),\n        try Value.createList(allocator, .{\n            Value.Null,\n            Value.integer(42),\n            Value.True,\n        }),\n    });\n    defer expected_value.unref();\n\n    try value.exepctEqual(expected_value);\n```\n\nThe decoders must be initialized with an allocator, which is used for internal encoder state. Each `decodeValue` call takes its own allocator that it uses for creating the actual Value elements, which does not have to be the same allocator used internally by the encoder.\n\n### Static Types\n\nInstead of allocating dynamic `Value` values, you can also decode directly into Zig types using the `decodeType` / `readType` decoder methods, and encode Zig types directly with `encodeType` / `writeType` decoder methods.\n\nThe patterns for Zig type mapping are as follows:\n\n0. `ipld.Value` types use the dynamic API; this is like an \"any\" type.\n1. booleans are IPLD Booleans\n2. integer types are IPLD Integers, and error when decoding an integer out of range\n3. float types are IPLD Floats\n4. slices, arrays, and tuple `struct` types are IPLD lists\n5. non-tuple `struct` types are IPLD Maps\n6. optional types are `null` for IPLD Null, and match the unwrapped child type otherwise\n7. for pointer types:\n   - encoding dereferences the pointer and encodes the child type\n   - decoding allocates an element with `allocator.create` and decodes the child type into it\n\nA simple encoding example looks like this\n\n```zig\nconst ipld = @import(\"ipld\");\nconst json = @import(\"dag-json\");\n\nconst Foo = struct { abc: u32, xyz: bool };\n\ntest \"encode a static type\" {\n    var encoder = json.Encoder.init(allocator, .{});\n    defer encoder.deinit();\n\n    const data = try encoder.encodeType(Foo, allocator, .{\n        .bar = 8,\n        .baz = false,\n    });\n    defer allocator.free(data);\n\n    try std.testing.expectEqualSlices(u8, \"{\\\"abc\\\":8,\\\"xyz\\\":false}\", data);\n}\n```\n\nFor decoding, `decodeType` returns a generic `Result(T)` struct that includes both the decoded `value: T` and a new `arena: ArenaAllocator` used for allocations within the value. This is unfortunately necessary since the decoder may encounter an error in the middle of decoding, and needs to be able to free the allocations in the partially-decoded value before returning the error to the user.\n\n```zig\nconst ipld = @import(\"ipld\");\nconst json = @import(\"dag-json\");\n\nconst Foo = struct { abc: u32, xyz: *const Bar };\nconst Bar = struct { id: u32, children: []const u32 };\n\ntest \"decode nested static types\" {\n    var decoder = json.Decoder.init(allocator, .{});\n    defer decoder.deinit();\n\n    const result = try encoder.decodeType(Foo, allocator,\n        \\\\{\"abc\":8,\"xyz\":{\"id\":9,\"children\":[1,2,10,87421]}}\"\n    );\n    defer result.deinit(); // calls result.arena.deinit() inline\n    // result: json.Decoder.Result(Foo) is a struct {\n    //     arena: std.heap.ArenaAllocator,\n    //     value: Foo,\n    // }\n\n    try std.testing.expectEqual(result.value.abc, 8);\n    try std.testing.expectEqual(result.value.xyz.id, 9);\n    try std.testing.expectEqualSlices(u32, result.value.xyz.children, \u0026.{1, 2, 10, 87421});\n}\n```\n\n### Strings and Bytes\n\nHandling strings and bytes is a point of unavoidable awkwardness. Idiomatic Zig generally uses `[]const u8` for both of these, but this is indistinguishable from \"an array of `u8`\" to zig-ipld. Furthermore, it's not possible to tell whether a user intends `[]const u8` to mean \"string\" or \"bytes\", and would be inappropriate to guess.\n\nThe simplest way to use strings and bytes in static types is to use the special struct types `String` and `Bytes` exported from the `ipld` module, which have a single `data: []const u8` field. Note that these are different than the dynamic values `Value.String` and `Value.Bytes`.\n\n```zig\nconst std = @import(\"std\");\nconst allocator = std.heap.c_allocator;\n\nconst ipld = @import(\"ipld\");\nconst json = @import(\"dag-json\");\n\nconst User = struct {\n    id: u32,\n    email: ipld.String,\n};\n\ntest \"encode static User\" {\n    var encoder = json.Encoder.init(allocator, .{});\n    defer encoder.deinit();\n\n    const bytes = try encoder.encodeType(User, allocator, .{\n        .id = 10,\n        .email = .{ .data = \"johndoe@example.com\" },\n    });\n    defer allocator.free(bytes);\n\n    try std.testing.expectEqualSlices(u8, bytes,\n    \\\\{\"email\":\"johndoe@example.com\",\"id\":10}\n    );\n}\n\ntest \"decode static User\" {\n    var decoder = json.Decoder.init(allocator, .{});\n    defer decoder.deinit();\n\n    const result = try decoder.decodeType(User, allocator,\n        \\\\{\"email\":\"johndoe@example.com\",\"id\":1}\n    );\n    defer result.deinit();\n\n    try std.testing.expectEqual(result.value.id, 1);\n    try std.testing.expectEqualSlices(u8, result.value.email.data, \"johndoe@example.com\");\n}\n```\n\nIf you want more flexibility, you can also add public function declarations to your struct/enum/union types to handle parsing to and from strings or bytes manually.\n\nTo represent a struct/enum/union as a string, add declarations\n\n- `pub fn parseIpldString(allocator: std.mem.Allocator, data: []const u8) !@This()`\n- `pub fn writeIpldString(self: @This(), writer: std.io.AnyWriter) !void`\n\nFor bytes, add\n\n- `pub fn parseIpldBytes(allocator: std.mem.Allocator, data: []const u8) !@This()`\n- `pub fn writeIpldBytes(self: @This(), writer: std.io.AnyWriter) !void`\n\nThese are what the `String` and `Bytes` structs internally. When parsing, they copy `data` using `allocator`, and when writing, they just call `writer.writeAll(self.data)`.\n\n```zig\npub const Bytes = struct {\n    data: []const u8,\n\n    pub fn parseIpldBytes(allocator: std.mem.Allocator, data: []const u8) !Bytes {\n        const copy = try allocator.alloc(u8, data.len);\n        @memcpy(copy, data);\n        return .{ .data = copy };\n    }\n\n    pub fn writeIpldBytes(self: Bytes, writer: std.io.AnyWriter) !void {\n        try writer.writeAll(self.data);\n    }\n};\n\npub const String = struct {\n    data: []const u8,\n\n    pub fn parseIpldString(allocator: std.mem.Allocator, data: []const u8) !String {\n        const copy = try allocator.alloc(u8, data.len);\n        @memcpy(copy, data);\n        return .{ .data = copy };\n    }\n\n    pub fn writeIpldString(self: String, writer: std.io.AnyWriter) !void {\n        try writer.writeAll(self.data);\n    }\n};\n```\n\nFor parsing, keep in mind that `allocator` is an arena allocator attached to the top-level `Result(T)`, so try to avoid making temporary allocations since they will be tied to the lifetime of the result even if you call `allocator.free(...)` etc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fndimensional%2Fzig-ipld","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fndimensional%2Fzig-ipld","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fndimensional%2Fzig-ipld/lists"}