{"id":17922587,"url":"https://github.com/rreverser/serde-ndim","last_synced_at":"2025-09-21T03:31:14.232Z","repository":{"id":152283681,"uuid":"622573349","full_name":"RReverser/serde-ndim","owner":"RReverser","description":"Serde support for n-dimensional arrays from self-describing formats","archived":false,"fork":false,"pushed_at":"2025-08-17T13:43:15.000Z","size":49,"stargazers_count":11,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-20T12:51:18.060Z","etag":null,"topics":["json","multidimensional-arrays","rust","rust-lang","rust-sci","scientific-computing","serde","serde-json"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RReverser.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-04-02T14:12:44.000Z","updated_at":"2025-08-17T13:43:17.000Z","dependencies_parsed_at":null,"dependency_job_id":"db6226a3-8379-4ad3-b257-3657d2261240","html_url":"https://github.com/RReverser/serde-ndim","commit_stats":{"total_commits":23,"total_committers":1,"mean_commits":23.0,"dds":0.0,"last_synced_commit":"0bf516c7d88345aa19fd47f4fd5a19b601cfa903"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/RReverser/serde-ndim","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RReverser%2Fserde-ndim","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RReverser%2Fserde-ndim/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RReverser%2Fserde-ndim/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RReverser%2Fserde-ndim/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RReverser","download_url":"https://codeload.github.com/RReverser/serde-ndim/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RReverser%2Fserde-ndim/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":276189900,"owners_count":25600282,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-21T02:00:07.055Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["json","multidimensional-arrays","rust","rust-lang","rust-sci","scientific-computing","serde","serde-json"],"created_at":"2024-10-28T20:39:53.349Z","updated_at":"2025-09-21T03:31:13.989Z","avatar_url":"https://github.com/RReverser.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# serde-ndim\n\n[![Crates.io](https://img.shields.io/crates/v/serde-ndim.svg)](https://crates.io/crates/serde-ndim)\n[![Documentation](https://docs.rs/serde-ndim/badge.svg)](https://docs.rs/serde-ndim)\n\n## Overview\n\nThis crate provides a way to serialize and deserialize arrays of arbitrary dimensionality from self-described formats such as JSON where no out-of-band data is provided about the shape of the resulting array.\n\nThis is useful for some data sources (e.g. in astronomical applications), but not the format supported by the built-in Serde integration of popular crates like `ndarray` or `nalgebra`.\n\nConsider input like the following:\n\n```json\n[\n    [\n        [1, 2, 3, 4],\n        [4, 5, 6, 7]\n    ],\n    [\n        [7, 8, 9, 10],\n        [10, 11, 12, 13]\n    ],\n    [\n        [13, 14, 15, 16],\n        [16, 17, 18, 19]\n    ]\n]\n```\n\nThis should deserialize into a 3-dimensional array of shape `[3, 2, 4]`. This crate provides `serialize` and `deserialize` functions that can be used via `#[serde(with = \"serde_ndim\")]` that do just that.\n\n## Deserialization\n\nThe tricky bit is that deserialization is built to learn and ensure internal consistency while reading the data:\n\n1. During the first descent, it waits until it reaches a leaf number (`1`) to determine number of dimensions from recursion depth (`3` in example above).\n2. It unwinds from the number one step up and reads the sequence `[1, 2, 3, 4]`, learning its length (`4`). Now it remembers the expected shape as `[unknown, unknown, 4]` - it hasn't seen the lengths of the upper dimensions, but at least it knows there are `3` dimensions and the last one has length `4`.\n3. It unwinds a step up, recurses into the next sequence, and reads `[4, 5, 6, 7]`. This time it knows it's not the first descent to this dimension, so instead of learning it, it validates the new length against the stored one (`4 == 4`, all good).\n4. It reached the end of this sequence of sequences, so now it knows and stores the expected shape as `[unknown, 2, 4]`.\n5. By repeating the process, it eventually learns and validates the shape of the whole array as `[3, 2, 4]`.\n6. All this time it was collecting raw numbers into a flat `Vec\u003c_\u003e` traditionally as an optimised storage of multidimensional arrays. Now it just needs to call a function that constructs a multidimensional array from the shape and flat data.\n\n**Note**: The resulting array will be in the standard column-major layout.\n\nConstructors for [`ndarray::Array`](https://docs.rs/ndarray/latest/ndarray/type.Array.html) and [`nalgebra::DMatrix`](https://docs.rs/nalgebra/latest/nalgebra/base/type.DMatrix.html) are provided out of the box under the `ndarray` and `nalgebra` features respectively, so you can use them like this:\n\n```rust\nuse serde::{Deserialize, Serialize};\n\n#[derive(Deserialize, Serialize)]\nstruct MyStruct {\n    #[serde(with = \"serde_ndim\")]\n    ndarray: ndarray::ArrayD\u003cf32\u003e,\n    /* ... */\n}\n```\n\nYou can also reuse deserialization for custom types by implementing the [`serde_ndarray::de::MakeNDim`](https://docs.rs/serde-ndim/latest/serde_ndim/de/trait.MakeNDim.html) trait.\n\n## Serialization\n\nSerialization is also provided. Its implementaton is much simpler, so I won't go into details here, feel free to check out the code if you want.\n\nIt's also provided for `ndarray::Array` and `nalgebra::DMatrix`, but if you want to serialize custom types, you can do so by implementing the [`serde_ndarray::ser::NDim`](https://docs.rs/serde-ndim/latest/serde_ndim/ser/trait.NDim.html) trait.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frreverser%2Fserde-ndim","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frreverser%2Fserde-ndim","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frreverser%2Fserde-ndim/lists"}