{"id":33931474,"url":"https://github.com/vincent-herlemont/native_model","last_synced_at":"2026-04-06T07:01:41.132Z","repository":{"id":192067267,"uuid":"686294497","full_name":"vincent-herlemont/native_model","owner":"vincent-herlemont","description":"A thin wrapper around serialized data which add information of identity and version.","archived":false,"fork":false,"pushed_at":"2026-02-03T18:08:39.000Z","size":232,"stargazers_count":30,"open_issues_count":9,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-18T09:43:27.831Z","etag":null,"topics":["data-consistency","data-model","decoding","deserialization","encoding","interoperability","performance","rust","serialization"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vincent-herlemont.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-09-02T10:13:30.000Z","updated_at":"2026-01-21T17:06:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"78e4839e-25e2-4bae-9a49-e98392c6d8bb","html_url":"https://github.com/vincent-herlemont/native_model","commit_stats":{"total_commits":108,"total_committers":5,"mean_commits":21.6,"dds":0.6111111111111112,"last_synced_commit":"10a5f37329810c87d4dbf590077f8d06ee714f32"},"previous_names":["vincent-herlemont/native_model"],"tags_count":63,"template":false,"template_full_name":null,"purl":"pkg:github/vincent-herlemont/native_model","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vincent-herlemont%2Fnative_model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vincent-herlemont%2Fnative_model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vincent-herlemont%2Fnative_model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vincent-herlemont%2Fnative_model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vincent-herlemont","download_url":"https://codeload.github.com/vincent-herlemont/native_model/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vincent-herlemont%2Fnative_model/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31463015,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T21:22:52.476Z","status":"online","status_checked_at":"2026-04-06T02:00:07.287Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-consistency","data-model","decoding","deserialization","encoding","interoperability","performance","rust","serialization"],"created_at":"2025-12-12T12:48:49.033Z","updated_at":"2026-04-06T07:01:41.125Z","avatar_url":"https://github.com/vincent-herlemont.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Native model\n\n[![Crates.io](https://img.shields.io/crates/v/native_model)](https://crates.io/crates/native_model)\n[![Build Test Release](https://github.com/vincent-herlemont/native_model/actions/workflows/build_and_test_release.yml/badge.svg)](https://github.com/vincent-herlemont/native_model/actions/workflows/build_and_test_release.yml)\n[![Documentation](https://docs.rs/native_model/badge.svg)](https://docs.rs/native_model)\n[![License](https://img.shields.io/crates/l/native_model)](LICENSE)\n\nAdd interoperability                  on the top  of   serialization formats like bincode, postcard etc.\n\nSee [concepts](#concepts) for more details.\n\n## Goals\n\n- **Interoperability**: Allows different applications to work together, even if they are using different\n  versions of the data model.\n- **Data Consistency**: Ensure that we process the data expected model.\n- **Flexibility**: You can use any serialization format you want. More details [here](#setup-your-serialization-format).\n- **Performance**: A minimal overhead (encode: ~20 ns, decode: ~40 ps). More details [here](#performance).\n\n## Usage\n\n```text\n       Application 1 (DotV1)        Application 2 (DotV1 and DotV2)\n                |                                  |\n   Encode DotV1 |--------------------------------\u003e | Decode DotV1 to DotV2\n                |                                  | Modify DotV2\n   Decode DotV1 | \u003c--------------------------------| Encode DotV2 back to DotV1\n                |                                  |\n```\n\n\n```rust\nuse native_model::native_model;\nuse serde::{Deserialize, Serialize};\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\n#[native_model(id = 1, version = 1)]\nstruct DotV1(u32, u32);\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\n#[native_model(id = 1, version = 2, from = DotV1)]\nstruct DotV2 {\n    name: String,\n    x: u64,\n    y: u64,\n}\n\nimpl From\u003cDotV1\u003e for DotV2 {\n    fn from(dot: DotV1) -\u003e Self {\n        DotV2 {\n            name: \"\".to_string(),\n            x: dot.0 as u64,\n            y: dot.1 as u64,\n        }\n    }\n}\n\nimpl From\u003cDotV2\u003e for DotV1 {\n    fn from(dot: DotV2) -\u003e Self {\n        DotV1(dot.x as u32, dot.y as u32)\n    }\n}\n\n// Application 1\nlet dot = DotV1(1, 2);\nlet bytes = native_model::encode(\u0026dot).unwrap();\n\n// Application 1 sends bytes to Application 2.\n\n// Application 2\n// We are able to decode the bytes directly into a new type DotV2 (upgrade).\nlet (mut dot, source_version) = native_model::decode::\u003cDotV2\u003e(bytes).unwrap();\nassert_eq!(dot, DotV2 {\n    name: \"\".to_string(),\n    x: 1,\n    y: 2\n});\ndot.name = \"Dot\".to_string();\ndot.x = 5;\n// For interoperability, we encode the data with the version compatible with Application 1 (downgrade).\nlet bytes = native_model::encode_downgrade(dot, source_version).unwrap();\n\n// Application 2 sends bytes to Application 1.\n\n// Application 1\nlet (dot, _) = native_model::decode::\u003cDotV1\u003e(bytes).unwrap();\nassert_eq!(dot, DotV1(5, 2));\n```\n\n - Full example [here](./tests_crate/tests/example/example_main.rs).\n\n## Serialization format\n\nYou can use  default serialization formats via  the feature flags, like:\n\n```toml\n[dependencies]\nnative_model = { version = \"0.1\", features = [\"bincode_2\"] }\n```\n\nEach feature flag corresponds to a specific minor version of the serialization format. In order to avoid breaking\nchanges, the default serialization format is the oldest one.\n\n- `bincode_1_3`: [bincode](https://docs.rs/bincode/1.3.3/bincode/) v1.3 (default)\n- `bincode_2`: [bincode](https://docs.rs/bincode/2.0.0-rc.3/bincode/) v2.0.0-rc3\n- `postcard_1_0`: [postcard](https://docs.rs/postcard/1.0.0/postcard/) v1.0\n- `rpm_serde_1_3`: [rmp-serde](https://docs.rs/rmp-serde/1.3.0/rmp_serde/) v1.3\n\n### Custom serialization format\n\nDefine a struct with the name you want. This struct must implement [`native_model::Encode`](https://docs.rs/native_model/latest/native_model/trait.Encode.html) and [`native_model::Decode`](https://docs.rs/native_model/latest/native_model/trait.Decode.html) traits.\n\nFull examples:\n- [bincode with encode/decode](./tests_crate/tests/example/custom_codec/bincode.rs)\n- [bincode with serde](./tests_crate/tests/example/custom_codec/bincode_serde.rs)\n\nOthers examples,  see the default implementations:\n- [bincode v1.3](./src/codec/bincode_1_3.rs)\n- [bincode v2.0 (rc)](./src/codec/bincode_2.rs)\n- [postcard v1.0](./src/codec/postcard_1_0.rs)\n- [rmp-serde v1.3](./src/codec/rmp_serde_1_3.rs)\n\n### Notice\n`native_model` provides implementations that rely on metadata-less formats and `serde`.\nThere are known issues with some `serde` advanced features such as:\n\n- `#[serde(flatten)]`\n- `#[serde(skip)]`\n- `#[serde(skip_deserializing)]`\n- `#[serde(skip_serializing)]`\n- `#[serde(skip_serializing_if = \"path\")]`\n- `#[serde(tag = \"...\")]`\n- `#[serde(untagged)]`\n\nOr types implementing similar strategies such as [`serde_json::Value`][serde_json_value].\n\nThe `rmp-serde` serialization format can optionally support them serializing structs as maps, the `RmpSerdeNamed` struct is provided to support this use-case.\n\n[serde_json_value]: https://docs.rs/serde_json/latest/serde_json/enum.Value.html\n\n## Data model\n\nDefine your model using the macro [`native_model`](file:///home/vincentherlemont/IdeaProjects/native_model/target/doc/native_model/attr.native_model.html).\n\nAttributes:\n- `id = u32`: The unique identifier of the model.\n- `version = u32`: The version of the model.\n- `with = type`: The serialization format that you use for the Encode/Decode implementation. Setup [here](#setup-your-serialization-format).\n- `from = type`: Optional, the previous version of the model.\n    - `type`: The previous version of the model that you use for the From implementation.\n- `try_from = (type, error)`: Optional, the previous version of the model with error handling.\n    - `type`: The previous version of the model that you use for the TryFrom implementation.\n    - `error`: The error type that you use for the TryFrom implementation.\n\n```rust\nuse native_model::native_model;\nuse serde::{Deserialize, Serialize};\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\n#[native_model(id = 1, version = 1)]\nstruct DotV1(u32, u32);\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\n#[native_model(id = 1, version = 2, from = DotV1)]\nstruct DotV2 {\n    name: String,\n    x: u64,\n    y: u64,\n}\n\n// Implement the conversion between versions From\u003cDotV1\u003e for DotV2 and From\u003cDotV2\u003e for DotV1.\n\nimpl From\u003cDotV1\u003e for DotV2 {\n    fn from(dot: DotV1) -\u003e Self {\n        DotV2 {\n            name: \"\".to_string(),\n            x: dot.0 as u64,\n            y: dot.1 as u64,\n        }\n    }\n}\n\nimpl From\u003cDotV2\u003e for DotV1 {\n    fn from(dot: DotV2) -\u003e Self {\n        DotV1(dot.x as u32, dot.y as u32)\n    }\n}\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\n#[native_model(id = 1, version = 3, try_from = (DotV2, anyhow::Error))]\nstruct DotV3 {\n    name: String,\n    cord: Cord,\n}\n\n#[derive(Deserialize, Serialize, PartialEq, Debug)]\nstruct Cord {\n    x: u64,\n    y: u64,\n}\n\n// Implement the conversion between versions From\u003cDotV2\u003e for DotV3 and From\u003cDotV3\u003e for DotV2.\n\nimpl TryFrom\u003cDotV2\u003e for DotV3 {\n    type Error = anyhow::Error;\n\n    fn try_from(dot: DotV2) -\u003e Result\u003cSelf, Self::Error\u003e {\n        Ok(DotV3 {\n            name: dot.name,\n            cord: Cord { x: dot.x, y: dot.y },\n        })\n    }\n}\n\nimpl TryFrom\u003cDotV3\u003e for DotV2 {\n    type Error = anyhow::Error;\n\n    fn try_from(dot: DotV3) -\u003e Result\u003cSelf, Self::Error\u003e {\n        Ok(DotV2 {\n            name: dot.name,\n            x: dot.cord.x,\n            y: dot.cord.y,\n        })\n    }\n}\n\n```\n\n## Codecs\n\n`native_model` comes with several optional built-in serializer features available:\n\n- [bincode 1.3](https://crates.io/crates/bincode/1.3.3)\n\t- This is the default codec.\n\t- **Warning: This codec may not work with all serde-derived types.**\n\n- [bincode 2.0.0-rc.3](https://crates.io/crates/bincode/2.0.0-rc.3)\n\t- Enable the `bincode_2` feature and use the `native_model::bincode_2::Bincode` attribute to have `native_db` use this crate for serializing \u0026 deserializing.\n\t- **Warning: This codec may not work with all serde-derived types.**\n\n- [postcard 1.0](https://crates.io/crates/postcard/1.0.8)\n\t- Enable the `postcard_1_0` feature and use the `native_model::postcard_1_0::PostCard` attribute.\n\t- **Warning: This codec may not work with all serde-derived types.**\n\n- [rmp-serde 1.3](https://crates.io/crates/rmp-serde/1.3.0)\n\t- Enable the `rmp_serde_1_3` feature and use the `native_model::rmp_serde_1_3::RmpSerde` attribute.\n\n###### Codec example:\n\nAs example, to use `rmp-serde`:\n\n1. In your project's `Cargo.toml` file, enable the `rmp_serde_1_3` feature for the `native_model` dependency.\n\t- Be sure to check `crates.io` for the most recent [`native_model`](https://crates.io/crates/native_model) version number.\n\n```toml\n[dependencies]\nserde = { version = \"1.0\", features = [ \"derive\" ] }\nnative_model = { version = \"0.4\", features = [ \"rmp_serde_1_3\" ] }\n```\n\n2. Assign the `rmp_serde_1_3` codec to your `struct` using the `with` attribute:\n\n```rust\nuse native_model::native_model;\n\n#[derive(Clone, Default, serde::Deserialize, serde::Serialize)]\n#[native_model(id = 1, version = 1, with = native_model::rmp_serde_1_3::RmpSerde)]\nstruct MyStruct {\n\tmy_string: String,\n\t// etc.\n}\n```\n\n###### Additional reading\n\nYou may also want to check out [David Koloski](https://github.com/djkoloski)'s [Rust serialization benchmarks](https://github.com/djkoloski/rust_serialization_benchmark) for help selecting the codec (i.e. `bincode_1_3`, `rmp_serde_1_3`, etc.) that's best for your project.\n\n## Status\n\nEarly development. Not ready for production.\n\n## Concepts\n\nIn order to understand how the native model works, you need to understand the following concepts.\n\n- **Identity**(`id`): The identity is the unique identifier of the model. It is used to identify the model and\n  prevent to decode a model into the wrong Rust type.\n- **Version**(`version`) The version is the version of the model. It is used to check the compatibility between two\n  models.\n- **Encode**: The encode is the process of converting a model into a byte array.\n- **Decode**: The decode is the process of converting a byte array into a model.\n- **Downgrade**: The downgrade is the process of converting a model into a previous version of the model.\n- **Upgrade**: The upgrade is the process of converting a model into a newer version of the model.\n\nUnder the hood, the native model is a thin wrapper around serialized data. The `id` and the `version` are twice encoded with a [`little_endian::U32`](https://docs.rs/zerocopy/latest/zerocopy/byteorder/little_endian/type.U32.html). That represents 8 bytes, that are added at the beginning of the data.\n\n``` text\n+------------------+------------------+------------------------------------+\n|     ID (4 bytes) | Version (4 bytes)| Data (indeterminate-length bytes)  |\n+------------------+------------------+------------------------------------+\n```\n\nFull example [here](tests/example/example_define_model.rs).\n\n## Performance\n\nNative model has\nbeen designed to have a minimal and constant overhead. That means that the overhead is the same\nwhatever the size of the data. Under the hood we use the [zerocopy](https://docs.rs/zerocopy/latest/zerocopy/) crate\nto avoid unnecessary copies.\n\n👉 To know the total time of the encode/decode, you need to add the time of your serialization format.\n\nResume:\n- **Encode**: ~20 ns\n- **Decode**: ~40 ps\n\n|      data size       |   encode time (ns)    | decode time (ps)        |\n|:--------------------:|:---------------------:|:-----------------------:|\n|         1 B          | 19.769 ns - 20.154 ns | 40.526 ps - 40.617 ps   |\n|        1 KiB         | 19.597 ns - 19.971 ns | 40.534 ps - 40.633 ps   |\n|        1 MiB         | 19.662 ns - 19.910 ns | 40.508 ps - 40.632 ps   |\n|        10 MiB        | 19.591 ns - 19.980 ns | 40.504 ps - 40.605 ps   |\n|       100 MiB        | 19.669 ns - 19.867 ns | 40.520 ps - 40.644 ps   |\n\nBenchmark of the native model overhead [here](benches/overhead.rs).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvincent-herlemont%2Fnative_model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvincent-herlemont%2Fnative_model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvincent-herlemont%2Fnative_model/lists"}