{"id":19602344,"url":"https://github.com/scottlamb/static-xml","last_synced_at":"2025-04-27T17:32:14.671Z","repository":{"id":44699335,"uuid":"438407889","full_name":"scottlamb/static-xml","owner":"scottlamb","description":"serde-like serialization and deserialization of static Rust types in XML","archived":false,"fork":false,"pushed_at":"2024-12-12T06:10:10.000Z","size":128,"stargazers_count":13,"open_issues_count":10,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-04-05T01:51:12.623Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/scottlamb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-12-14T21:40:53.000Z","updated_at":"2024-12-12T06:10:15.000Z","dependencies_parsed_at":"2022-09-12T13:41:18.129Z","dependency_job_id":null,"html_url":"https://github.com/scottlamb/static-xml","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fstatic-xml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fstatic-xml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fstatic-xml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/scottlamb%2Fstatic-xml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/scottlamb","download_url":"https://codeload.github.com/scottlamb/static-xml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251178005,"owners_count":21548144,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T09:23:37.243Z","updated_at":"2025-04-27T17:32:14.399Z","avatar_url":"https://github.com/scottlamb.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# static-xml\n\n`static-xml` is a [`serde`](https://serde.rs/)-like serialization and\ndeserialization library for XML, currently written as a layer on top of\n[`xml-rs`](https://crates.io/crates/xml-rs). It is *inspired by* `serde`\nbut is an entirely separate implementation rather than a `serde` data format.\n\n*Status:* in early development. API is unstable.\n\n## Goals\n\nIn order, roughly:\n\n1.  **sound.** `unsafe` is allowed, but never in a way that allows safe calling\n    code to violate soundness. It should be carefully reasoned and thoroughly\n    tested using tools such as `cargo miri test`.\n2.  **correct.** It should have a bug-free mapping of Rust types and XML\n    documents with no surprises. The APIs should be misuse-resistant.\n3.  **ergonomic.** The derive macros should be easy to use correctly, and they\n    should produce good error messages when used incorrectly.\n4.  **complete enough.** The `static-xml` crate should support any useful\n    format; the `static-xml-derive` macros should support most types.\n    See design notes below.\n5.  **light-weight.** It should produce reasonably small binaries and compile\n    quickly. This is a major concern for large XML schemas such as ONVIF.\n6.  **fast enough.** It should be as fast as possible without compromising\n    the goals above. The lowest-hanging fruit for improving performance are\n    likely in `xml-rs`, not in `static-xml` itself.\n\n## Design notes\n\nThis library is divided into two crates:\n\n1.  `static-xml` has the main logic for driving serialization and\n    deserialization, the basic traits, and a few helpers for\n    `Serialize`/`Deserialize` impls to use to reduce code size.\n\n    `static-xml` is meant to support any useful format. Its current\n    implementation has some limitations. E.g. it doesn't handle processing\n    instructions. In theory, though, it could be extended to exactly round-trip\n    any series of XML events via the underlying XML library.\n\n2.  `static-xml-derive` has macros for automatically deriving `Serialize` and\n    `Deserialize` impls on `struct` and `enum` types. It's designed to work\n    with simple Rust types that can convey the semantic meaning of typical XML\n    types but often will not round-trip to the exact same events or bytes. You\n    may need to bypass it for some types. For example, the `struct`s it supports\n    don't have any way of conveying order between their fields. To make this\n    concrete, when fed an XHTML document, deserialization would lose the\n    distinction between `\u003cp\u003efoo\u003ci\u003ebar\u003c/i\u003ebaz\u003c/p\u003e` and `\u003cp\u003efoobaz\u003ci\u003ebar\u003c/i\u003e\u003c/p\u003e`.\n    Similarly, it doesn't support validating all the rules that might be\n    expressed in an XML schema. It can parse a schema described via\n    `\u003cxs:sequence\u003e`, but it won't produce an error if the elements aren't\n    written in the stated order.\n\n### `xml-rs` vs `quick-xml` or other alternatives\n\nThis library is written on top of `xml-rs`'s stream-of-events interface.\nI considered other libraries like `quick-xml` but chose `xml-rs` for a couple\nof reasons:\n\n1.  It's the most widely used, and there are companion crates like `xmltree`\n    available.\n2.  It aims to be standards-compliant, and others don't.\n\n`xml-rs` is not without problems. E.g. its author [wrote](https://github.com/netvl/xml-rs/issues/126#issuecomment-374107131)\nthat \"xml-rs has been first created ages ago, long before the first stable\nversion of Rust was available. Therefore some details of its API are not really\nup-to-date. In particular, xml-rs allocates a lot. Ideally, it should work like\nquick-xml does, i.e. reading data to its internal buffer and give out references\nto it.\"\n\nI'm open to porting `static-xml` to another library if there's one that aims for\nreasonably good standards compliance, or to a new `xml-rs` version if someone\ntakes on the task of freshening that crate.\n\n`static-xml` could even support multiple underlying XML crates via feature\nflags. The library makes use of `\u0026dyn Trait` indirection internally, so\nadditional code bloat should be minimal. Some interface choices borrowed from\n`xml-rs` would have to change to take the most advantage of an underlying\nlibrary that allocates less.\n\n### DOM tree support\n\nIt's possible to deserialize Rust types from an in-memory DOM tree rather than\nXML events (and vice versa). This was suggested in [this\ncomment](https://github.com/media-io/yaserde/issues/76#issuecomment-646947783).\nI don't believe this simplifies the implementation much: it's beneficial anyway\nto use the program stack to represent the types during deserialization.\n\nThe streaming interface is strictly more general: just as it's possible for\n`static-xml` to support multiple underlying XML streaming libraries, it could\nalso support traversing a DOM tree.\n\nIn the other direction, I plan to add an `xmltree` feature to `static-xml` which\nsupplies a `Serialize` and `Deserialize` impl on `xmltree::Element`. This would\nallow retaining unknown field values easily:\n\n```rust\n#[derive(Deserialize, Serialize)]\nstruct Foo {\n    known_field: String,\n    #[static_xml(flatten)]\n    unknown_fields: xmltree::Element,\n}\n```\n\n### Future work: table-driven `Visitor` impl.\n\nCurrently `static-xml-derive` writes explicit generated code. E.g., the\n`Deserialize` impl for `Foo` above looks roughly as follows:\n\n```rust\nconst ELEMENTS: \u0026[ExpandedNameRef; 1] = \u0026[\n    ExpandedNameRef { local_name: \"known_field\", namespace: \"\" },\n];\n\nimpl Deserialize for Foo {\n    fn deserialize(element: ElementReader\u003c'_\u003e) -\u003e Result\u003cSelf, VisitorError\u003e {\n        let mut builder = FooVisitor {\n            known_field: \u003cString as DeserializeField\u003e::init(),\n            unknown_fields: \u003cXmlTree as DeserializeField\u003e::init(),\n        };\n        element.read_to(\u0026mut builder)?;\n        Self {\n            known_field: \u003cString as DeserializeField\u003e::finalize(builder.known_field)?,\n            unknown_fields: \u003cXmlTree as DeserializeField\u003e::finalize(builder.unknown_fields)?,\n        }\n    }\n}\n\npub struct FooVisitor {\n    known_field: \u003cString as DeserializeField\u003e::Builder,\n    unknown_fields: \u003cxmltree::Element as DeserializeFlatten\u003e::Builder,\n}\n\nimpl ElementVisitor for FooVisitor {\n    fn element\u003c'a\u003e(\n        \u0026mut self,\n        child: ElementReader\u003c'a\u003e\n    ) -\u003e Result\u003cOption\u003cElementReader\u003c'a\u003e\u003e, VisitorError\u003e {\n        match find(\u0026child.expanded_name(), ELEMENTS) {\n            Some(0usize) =\u003e {\n                ::static_xml::de::DeserializeFieldBuilder::element(\u0026mut self.known_field, child)?;\n                return Ok(None);\n            }\n            _ =\u003e delegate_element(\u0026mut [\u0026mut self.unknown_fields], child),\n        }\n    }\n}\n```\n\nI believe this is close to the minimal size with this approach. Next I'd like\nto experiment with a different approach in which the `Visitor` impl is replaced\nwith a table that holds the offset within `FooVisitor` of each field, and a\npointer to an `element` function. The generated code would use `unsafe`, but\nsoundness only has to be proved once in the generator, and this seems worthwhile\nif it can achieve significant code size reduction. See\n[#5](https://github.com/scottlamb/static-xml/issues/5).\n\n## Comparison with other crates\n\n### `static-xml` vs a `serde` data format\n\nThere are several XML serialization crates that plug into `serde` as a data\nformat (`serde::Deserializer` and `serde::Serializer` impls), including:\n\n*   [`serde-xml-rs`](https://github.com/RReverser/serde-xml-rs) (most popular)\n*   [`xml_serde`](https://crates.io/crates/xml_serde) (most capable)\n\nThis is an attractive idea: take advantage of `serde`'s high-quality derive\nmacro implementation and maybe even a few existing `#[derive(Serialize)`\nannotations in popular crates.\n\nI discarded this approach because I found it frustrating to combine [serde's\ngeneric data model](https://serde.rs/data-model.html) and XML's complex, unique\ndata model. The challenge is to make it possible to use [serde\nattributes](https://serde.rs/attributes.html) to describe an XML data format\neasily:\n\n*   on [`serde::de::Deserializer`](`https://docs.serde.rs/serde/de/trait.Deserializer.html`) calls,\n    turn ([`xml::reader::XmlEvent`](https://docs.rs/xml-rs/0.8.4/xml/reader/enum.XmlEvent.html))s\n    into [`serde::de::Visitor`](https://docs.serde.rs/serde/de/trait.Visitor.html) calls.\n*   on [`serde::ser::Serializer`](https://docs.serde.rs/serde/ser/trait.Serializer.html) calls,\n    produce [`xml::writer::events::XmlEvent`](https://docs.rs/xml-rs/0.8.4/xml/writer/events/enum.XmlEvent.html)s.\n\nA few examples of the mismatch:\n\n*   XML distinguishes between elements and attributes. `serde-xml-rs`\n    [doesn't support attributes](see (https://github.com/RReverser/serde-xml-rs/issues/140)).\n    `xml-serde` uses a special `$attr:` rename prefix.\n*   XML not only is namespaced but does so indirectly, by assigning prefixes to\n    namespaces and referencing prefixes in element and\n    attribute names. `serde-xml-rs` [doesn't support\n    namespaces](https://github.com/RReverser/serde-xml-rs/issues/50).\n    `xml-serde` uses a `{namespace}prefix:element` name for every field, which\n    can be verbose both in the struct definition and the generated XML\n    (not supporting binding a prefix at a higher level than it is used).\n*   Even XML schema's \"simple types\" (the strings within text nodes\n    and attribute values) can be quite complex:\n    *   They can represent a list of values separated by spaces. The deserializer\n        might be able to hint it's expecting this by calling eg `deserialize_seq`\n        rather than `deserialize_string`.\n    *   They can represent a \"union\": any of several possible subtypes. Now we\n        need to support accumulating them in some buffer and backtracking.\n        The buffer needs to also support these deserializer hints. The caller\n        likely needs to request this buffering in some fashion, likely by\n        wrapping with a type from this library, dropping the `serde` data format\n        abstraction.\n    *   They support three modes of [whitespace\n        normalization](https://www.w3.org/TR/xmlschema11-1/#sec-wsnormalization).\n        There's no way to pass this through serde, other than custom types or\n        `#[serde(deserialize_with)]` functions.\n\nThese problems can likely be solved, but I find it much easier to understand a\ndata model specific to XML. It can be extended to support as much of XML as\nnecessary without wedging a square peg into a round hole.\n\n### `static-xml` vs `yaserde`\n\n[`yaserde`](https://crates.io/crates/yaserde) is conceptually similar to\n`static-xml` but suffers from poor implementation quality.\n\n#### Error handling\n\n`yaserde`'s generated code will panic on invalid data, e.g. if a non-digit\nis found where an `i32` is expected:\n\n```\nthread 'tests::basic_deserialization' panicked at 'called `Result::unwrap()` on an `Err` value: ParseIntError { kind: InvalidDigit }', schema/src/onvif.rs:4030:50\n```\n\n`static-xml` instead returns a nicely formatted error:\n\n```\ninvalid digit found in string @ 14:25\n\nXML element stack:\n   4: \u003ctt:Hour\u003e @ 14:25\n   3: \u003ctt:Time\u003e @ 13:21\n   2: \u003ctt:UTCDateTime\u003e @ 12:17\n   1: \u003ctds:SystemDateAndTime\u003e @ 6:13\n   0: \u003ctds:GetSystemDateAndTimeResponse\u003e @ 3:9\n```\n\n#### Bugs\n\n`yaserde` has several variations of unsolved bugs involving nested elements with\nthe same name (eg [#76](https://github.com/media-io/yaserde/issues/76)). The\nroot cause is that it doesn't have a well-defined contract for the\ndeserialization interface and doesn't track the depth reliably.\n\n`static-xml` is based on the proposal in [#84](https://github.com/media-io/yaserde/issues/84)\nwhich solves these problems systematically, introducing a deserialization\ncontract which is enforced by Rust's type system.\n\n`yaserde` also has several bugs involving namespaces, eg\n[#126](https://github.com/media-io/yaserde/issues/126), and enum element name\ncomparisons ignoring the namespace entirely. These are believed to be addressed\nby `static-xml`, although many tests have yet to be written.\n\n#### Bloat\n\nWith large schemas, `yaserde` bloats binaries and compilation time. Using\n[`lumeohq/onvif-rs`](https://github.com/lumeohq/onvif-rs) 247b90c and Rust,\nlook at the code sizes below, particularly for the `schema` crate that contains\nyaserde's generated code.\n\n```\n$ cargo bloat --release --example camera --crates\n...\n File  .text      Size Crate\n 7.5%  21.4% 1016.4KiB schema\n 4.8%  13.7%  654.1KiB std\n 4.1%  11.7%  556.6KiB reqwest\n 3.3%   9.3%  443.9KiB yaserde\n 2.2%   6.2%  295.5KiB clap\n 1.6%   4.4%  211.6KiB h2\n 1.2%   3.4%  160.4KiB regex_syntax\n 1.1%   3.1%  147.7KiB onvif\n 1.0%   2.9%  139.0KiB tokio\n 0.9%   2.6%  124.6KiB hyper\n 0.9%   2.6%  122.7KiB tracing_subscriber\n 0.6%   1.8%   83.7KiB regex_automata\n 0.6%   1.7%   80.8KiB xml\n 0.5%   1.5%   72.8KiB regex\n 0.4%   1.1%   52.4KiB http\n 0.4%   1.0%   48.9KiB url\n 0.3%   0.9%   43.7KiB num_bigint\n 0.3%   0.9%   42.9KiB chrono\n 0.3%   0.8%   36.8KiB idna\n 0.3%   0.7%   35.4KiB encoding_rs\n 2.3%   6.4%  305.0KiB And 62 more crates. Use -n N to show more.\n35.2% 100.0%    4.6MiB .text section size, the file size is 13.2MiB\n\nNote: numbers above are a result of guesswork. They are not 100% correct and never will be.\n```\n\nCompare to numbers from a WIP branch based on `static-xml` (which are likely to\nfurther improve):\n\n```\n File  .text     Size Crate\n 5.0%  17.1% 655.0KiB std\n 4.3%  14.5% 557.9KiB reqwest\n 2.3%   7.7% 295.5KiB clap\n 2.2%   7.4% 282.4KiB schema\n 1.7%   5.7% 218.5KiB regex\n 1.6%   5.5% 211.6KiB h2\n 1.4%   4.8% 185.9KiB regex_syntax\n 1.1%   3.7% 142.1KiB tokio\n 1.0%   3.3% 125.1KiB tracing_subscriber\n 1.0%   3.2% 124.3KiB hyper\n 0.8%   2.7% 104.2KiB onvif\n 0.6%   2.2%  82.8KiB regex_automata\n 0.6%   1.9%  73.8KiB aho_corasick\n 0.5%   1.8%  68.7KiB xml\n 0.5%   1.7%  65.9KiB static_xml\n 0.4%   1.4%  52.0KiB http\n 0.4%   1.3%  48.9KiB url\n 0.3%   1.1%  43.8KiB num_bigint\n 0.3%   1.1%  42.9KiB chrono\n 0.3%   1.0%  36.8KiB idna\n 2.5%   8.5% 327.0KiB And 63 more crates. Use -n N to show more.\n29.6% 100.0%   3.7MiB .text section size, the file size is 12.7MiB\n```\n\nOn a powerful 12-core/24-thread AMD Ryzen 5900X machine,\n`cargo bloat --release --example camera --times` says the `yaserde`-based\n`schema` crate takes 97.99s to compile; the `static-xml`-based version takes\n33.43s to compile. The difference is even more dramatic on older machines.\nOn several of my SBC setups, the `yaserde` version fails to compile\nwithout enabling zramfs.\n\n#### Compile-time errors\n\n`yaserde`'s derive macros will panic in some cases with an unhelpful error\nmessage. In others, they emit code that doesn't compile and doesn't have the\nproper spans. Eg, if a field doesn't implement the required `YaSerialize`\ninterface, it describes the problem but doesn't pinpoint the offending line of\ncode:\n\n```\nerror[E0277]: the trait bound `Foo: YaSerialize` is not satisfied\n   --\u003e schema/src/common.rs:38:37\n    |\n38  | #[derive(Default, PartialEq, Debug, YaSerialize, YaDeserialize)]\n    |                                     ^^^^^^^^^^^ the trait `YaSerialize` is not implemented for `Foo`\n    |\nnote: required by a bound in `yaserde::YaSerialize::serialize`\n   --\u003e /home/slamb/.cargo/registry/src/github.com-1ecc6299db9ec823/yaserde-0.7.1/src/lib.rs:106:19\n    |\n106 |   fn serialize\u003cW: Write\u003e(\u0026self, writer: \u0026mut ser::Serializer\u003cW\u003e) -\u003e Result\u003c(), String\u003e;\n    |                   ^^^^^ required by this bound in `yaserde::YaSerialize::serialize`\n    = note: this error originates in the derive macro `YaSerialize` (in Nightly builds, run with -Z macro-backtrace for more info)\n```\n\n`static-xml`'s derive macros always try to add a relevant span.\n\n```\nerror[E0277]: the trait bound `Foo: ParseText` is not satisfied\n  --\u003e schema/src/common.rs:49:9\n   |\n49 |     pub foo: Foo,\n   |         ^^^ the trait `ParseText` is not implemented for `Foo`\n   |\n   = note: required because of the requirements on the impl of `Deserialize` for `Foo`\n   = note: required because of the requirements on the impl of `DeserializeFieldBuilder` for `Foo`\n   = help: see issue #48214\n```\n\n#### Flexibility\n\n`yaserde` requires that every deserializable type implement `Default`, which is\nparticularly awkward for `enum`s. It also doesn't support required fields or\ndistinguishing between absent fields and ones set to the default value.\n\n`static-xml` avoids this by defining a builder type matching each deserializable\ntype. Some caveats apply: currently the builders' `finalize` methods are a\nsignificant source of code bloat, so there's a `direct` knob to turn them off.\nI'd like to see if I can reduce the bloat without giving up the builders'\nadvantages.\n\n## License\n\nYour choice of MIT or Apache; see [LICENSE-MIT.txt](LICENSE-MIT.txt) or\n[LICENSE-APACHE](LICENSE-APACHE.txt), respectively.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlamb%2Fstatic-xml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fscottlamb%2Fstatic-xml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fscottlamb%2Fstatic-xml/lists"}