{"id":13413954,"url":"https://github.com/clbanning/mxj","last_synced_at":"2025-03-14T20:30:50.638Z","repository":{"id":13785198,"uuid":"16480442","full_name":"clbanning/mxj","owner":"clbanning","description":"Decode / encode XML to/from map[string]interface{} (or JSON); extract values with dot-notation paths and wildcards.  Replaces x2j and j2x packages.","archived":false,"fork":false,"pushed_at":"2024-04-03T21:58:22.000Z","size":2177,"stargazers_count":616,"open_issues_count":2,"forks_count":111,"subscribers_count":25,"default_branch":"master","last_synced_at":"2024-07-31T20:53:10.848Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"tensorflow/tensorflow","license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/clbanning.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-02-03T13:39:16.000Z","updated_at":"2024-07-30T08:01:59.000Z","dependencies_parsed_at":"2024-06-18T11:29:01.791Z","dependency_job_id":null,"html_url":"https://github.com/clbanning/mxj","commit_stats":{"total_commits":526,"total_committers":23,"mean_commits":"22.869565217391305","dds":0.5247148288973384,"last_synced_commit":"cc007f32c523caac701590a6b30200ee8d573176"},"previous_names":[],"tags_count":34,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clbanning%2Fmxj","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clbanning%2Fmxj/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clbanning%2Fmxj/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/clbanning%2Fmxj/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/clbanning","download_url":"https://codeload.github.com/clbanning/mxj/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243642012,"owners_count":20323949,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:53.576Z","updated_at":"2025-03-14T20:30:50.137Z","avatar_url":"https://github.com/clbanning.png","language":"Go","funding_links":[],"categories":["Text Processing","文本处理","Template Engines","文本处理`解析和操作文本的代码库`","Bot Building","文本處理","Specific Formats","\u003cspan id=\"文字处理-text-processing\"\u003e文字处理 Text Processing\u003c/span\u003e"],"sub_categories":["Markup Languages","Advanced Console UIs","标记语言","查询语","HTTP Clients","交流","高級控制台界面","高级控制台界面","\u003cspan id=\"高级控制台用户界面-advanced-console-uis\"\u003e高级控制台用户界面 Advanced Console UIs\u003c/span\u003e"],"readme":"\u003ch2\u003emxj - to/from maps, XML and JSON\u003c/h2\u003e\nDecode/encode XML to/from map[string]interface{} (or JSON) values, and extract/modify values from maps by key or key-path, including wildcards.\n\nmxj supplants the legacy x2j and j2x packages. If you want the old syntax, use mxj/x2j and mxj/j2x packages.\n\n\u003ch4\u003eInstallation\u003c/h4\u003e\nUsing go.mod:\n\u003cpre\u003e\ngo get github.com/clbanning/mxj/v2@v2.7\t\n\u003c/pre\u003e\n\n\u003cpre\u003e\nimport \"github.com/clbanning/mxj/v2\"\n\u003c/pre\u003e\n\n... or just vendor the package.\n\n\u003ch4\u003eRelated Packages\u003c/h4\u003e\n\nhttps://github.com/clbanning/checkxml provides functions for validating XML data.\n\n\u003ch4\u003eRefactor Encoder - 2020.05.01\u003c/h4\u003e\nIssue #70 highlighted that encoding large maps does not scale well, since the original logic used string appends operations. Using bytes.Buffer results in linear scaling for very large XML docs. (Metrics based on MacBook Pro i7 w/ 16 GB.)\n\n\tNodes      m.XML() time\n\t54809       12.53708ms\n\t109780      32.403183ms\n\t164678      59.826412ms\n\t482598     109.358007ms\n\n\u003ch4\u003eRefactor Decoder - 2015.11.15\u003c/h4\u003e\nFor over a year I've wanted to refactor the XML-to-map[string]interface{} decoder to make it more performant.  I recently took the time to do that, since we were using github.com/clbanning/mxj in a production system that could be deployed on a Raspberry Pi.  Now the decoder is comparable to the stdlib JSON-to-map[string]interface{} decoder in terms of its additional processing overhead relative to decoding to a structure value.  As shown by:\n\n\tBenchmarkNewMapXml-4         \t  100000\t     18043 ns/op\n\tBenchmarkNewStructXml-4      \t  100000\t     14892 ns/op\n\tBenchmarkNewMapJson-4        \t  300000\t      4633 ns/op\n\tBenchmarkNewStructJson-4     \t  300000\t      3427 ns/op\n\tBenchmarkNewMapXmlBooks-4    \t   20000\t     82850 ns/op\n\tBenchmarkNewStructXmlBooks-4 \t   20000\t     67822 ns/op\n\tBenchmarkNewMapJsonBooks-4   \t  100000\t     17222 ns/op\n\tBenchmarkNewStructJsonBooks-4\t  100000\t     15309 ns/op\n\n\u003ch4\u003eNotices\u003c/h4\u003e\n\n\t2022.11.28: v2.7 - add SetGlobalKeyMapPrefix to change default prefix, '#', for default keys\n\t2022.11.20: v2.6 - add NewMapForattedXmlSeq for XML docs formatted with whitespace character\n\t2021.02.02: v2.5 - add XmlCheckIsValid toggle to force checking that the encoded XML is valid\n\t2020.12.14: v2.4 - add XMLEscapeCharsDecoder to preserve XML escaped characters in Map values\n\t2020.10.28: v2.3 - add TrimWhiteSpace option\n\t2020.05.01: v2.2 - optimize map to XML encoding for large XML docs.\n\t2019.07.04: v2.0 - remove unnecessary methods - mv.XmlWriterRaw, mv.XmlIndentWriterRaw - for Map and MapSeq.\n\t2019.07.04: Add MapSeq type and move associated functions and methods from Map to MapSeq.\n\t2019.01.21: DecodeSimpleValuesAsMap - decode to map[\u003ctag\u003e:map[\"#text\":\u003cvalue\u003e]] rather than map[\u003ctag\u003e:\u003cvalue\u003e]\n\t2018.04.18: mv.Xml/mv.XmlIndent encodes non-map[string]interface{} map values - map[string]string, map[int]uint, etc.\n\t2018.03.29: mv.Gob/NewMapGob support gob encoding/decoding of Maps.\n\t2018.03.26: Added mxj/x2j-wrapper sub-package for migrating from legacy x2j package.\n\t2017.02.22: LeafNode paths can use \".N\" syntax rather than \"[N]\" for list member indexing.\n\t2017.02.10: SetFieldSeparator changes field separator for args in UpdateValuesForPath, ValuesFor... methods.\n\t2017.02.06: Support XMPP stream processing - HandleXMPPStreamTag().\n\t2016.11.07: Preserve name space prefix syntax in XmlSeq parser - NewMapXmlSeq(), etc.\n\t2016.06.25: Support overriding default XML attribute prefix, \"-\", in Map keys - SetAttrPrefix().\n\t2016.05.26: Support customization of xml.Decoder by exposing CustomDecoder variable.\n\t2016.03.19: Escape invalid chars when encoding XML attribute and element values - XMLEscapeChars().\n\t2016.03.02: By default decoding XML with float64 and bool value casting will not cast \"NaN\", \"Inf\", and \"-Inf\".\n\t            To cast them to float64, first set flag with CastNanInf(true).\n\t2016.02.22: New mv.Root(), mv.Elements(), mv.Attributes methods let you examine XML document structure.\n\t2016.02.16: Add CoerceKeysToLower() option to handle tags with mixed capitalization.\n\t2016.02.12: Seek for first xml.StartElement token; only return error if io.EOF is reached first (handles BOM).\n\t2015.12.02: XML decoding/encoding that preserves original structure of document. See NewMapXmlSeq()\n\t            and mv.XmlSeq() / mv.XmlSeqIndent().\n\t2015-05-20: New: mv.StringIndentNoTypeInfo().\n\t            Also, alphabetically sort map[string]interface{} values by key to prettify output for mv.Xml(),\n\t            mv.XmlIndent(), mv.StringIndent(), mv.StringIndentNoTypeInfo().\n\t2014-11-09: IncludeTagSeqNum() adds \"_seq\" key with XML doc positional information.\n\t            (NOTE: PreserveXmlList() is similar and will be here soon.)\n\t2014-09-18: inspired by NYTimes fork, added PrependAttrWithHyphen() to allow stripping hyphen from attribute tag.\n\t2014-08-02: AnyXml() and AnyXmlIndent() will try to marshal arbitrary values to XML.\n\t2014-04-28: ValuesForPath() and NewMap() now accept path with indexed array references.\n\n\u003ch4\u003eBasic Unmarshal XML to map[string]interface{}\u003c/h4\u003e\n\u003cpre\u003etype Map map[string]interface{}\u003c/pre\u003e\n\nCreate a `Map` value, 'mv', from any `map[string]interface{}` value, 'v':\n\u003cpre\u003emv := Map(v)\u003c/pre\u003e\n\nUnmarshal / marshal XML as a `Map` value, 'mv':\n\u003cpre\u003emv, err := NewMapXml(xmlValue) // unmarshal\nxmlValue, err := mv.Xml()      // marshal\u003c/pre\u003e\n\nUnmarshal XML from an `io.Reader` as a `Map` value, 'mv':\n\u003cpre\u003emv, err := NewMapXmlReader(xmlReader)         // repeated calls, as with an os.File Reader, will process stream\nmv, raw, err := NewMapXmlReaderRaw(xmlReader) // 'raw' is the raw XML that was decoded\u003c/pre\u003e\n\nMarshal `Map` value, 'mv', to an XML Writer (`io.Writer`):\n\u003cpre\u003eerr := mv.XmlWriter(xmlWriter)\nraw, err := mv.XmlWriterRaw(xmlWriter) // 'raw' is the raw XML that was written on xmlWriter\u003c/pre\u003e\n   \nAlso, for prettified output:\n\u003cpre\u003exmlValue, err := mv.XmlIndent(prefix, indent, ...)\nerr := mv.XmlIndentWriter(xmlWriter, prefix, indent, ...)\nraw, err := mv.XmlIndentWriterRaw(xmlWriter, prefix, indent, ...)\u003c/pre\u003e\n\nBulk process XML with error handling (note: handlers must return a boolean value):\n\u003cpre\u003eerr := HandleXmlReader(xmlReader, mapHandler(Map), errHandler(error))\nerr := HandleXmlReaderRaw(xmlReader, mapHandler(Map, []byte), errHandler(error, []byte))\u003c/pre\u003e\n\nConverting XML to JSON: see Examples for `NewMapXml` and `HandleXmlReader`.\n\nThere are comparable functions and methods for JSON processing.\n\nArbitrary structure values can be decoded to / encoded from `Map` values:\n\u003cpre\u003emv, err := NewMapStruct(structVal)\nerr := mv.Struct(structPointer)\u003c/pre\u003e\n\n\u003ch4\u003eExtract / modify Map values\u003c/h4\u003e\nTo work with XML tag values, JSON or Map key values or structure field values, decode the XML, JSON\nor structure to a `Map` value, 'mv', or cast a `map[string]interface{}` value to a `Map` value, 'mv', then:\n\u003cpre\u003epaths := mv.PathsForKey(key)\npath := mv.PathForKeyShortest(key)\nvalues, err := mv.ValuesForKey(key, subkeys)\nvalues, err := mv.ValuesForPath(path, subkeys)\ncount, err := mv.UpdateValuesForPath(newVal, path, subkeys)\u003c/pre\u003e\n\nGet everything at once, irrespective of path depth:\n\u003cpre\u003eleafnodes := mv.LeafNodes()\nleafvalues := mv.LeafValues()\u003c/pre\u003e\n\nA new `Map` with whatever keys are desired can be created from the current `Map` and then encoded in XML\nor JSON. (Note: keys can use dot-notation.)\n\u003cpre\u003enewMap, err := mv.NewMap(\"oldKey_1:newKey_1\", \"oldKey_2:newKey_2\", ..., \"oldKey_N:newKey_N\")\nnewMap, err := mv.NewMap(\"oldKey1\", \"oldKey3\", \"oldKey5\") // a subset of 'mv'; see \"examples/partial.go\"\nnewXml, err := newMap.Xml()   // for example\nnewJson, err := newMap.Json() // ditto\u003c/pre\u003e\n\n\u003ch4\u003eUsage\u003c/h4\u003e\n\nThe package is fairly well [self-documented with examples](http://godoc.org/github.com/clbanning/mxj).\n\nAlso, the subdirectory \"examples\" contains a wide range of examples, several taken from golang-nuts discussions.\n\n\u003ch4\u003eXML parsing conventions\u003c/h4\u003e\n\nUsing NewMapXml()\n\n   - Attributes are parsed to `map[string]interface{}` values by prefixing a hyphen, `-`,\n     to the attribute label. (Unless overridden by `PrependAttrWithHyphen(false)` or\n     `SetAttrPrefix()`.)\n   - If the element is a simple element and has attributes, the element value\n     is given the key `#text` for its `map[string]interface{}` representation.  (See\n     the 'atomFeedString.xml' test data, below.)\n   - XML comments, directives, and process instructions are ignored.\n   - If CoerceKeysToLower() has been called, then the resultant keys will be lower case.\n\nUsing NewMapXmlSeq()\n\n   - Attributes are parsed to `map[\"#attr\"]map[\u003cattr_label\u003e]map[string]interface{}`values\n     where the `\u003cattr_label\u003e` value has \"#text\" and \"#seq\" keys - the \"#text\" key holds the \n     value for `\u003cattr_label\u003e`.\n   - All elements, except for the root, have a \"#seq\" key.\n   - Comments, directives, and process instructions are unmarshalled into the Map using the\n     keys \"#comment\", \"#directive\", and \"#procinst\", respectively. (See documentation for more\n     specifics.)\n   - Name space syntax is preserved: \n      - `\u003cns:key\u003esomething\u003c/ns.key\u003e` parses to `map[\"ns:key\"]interface{}{\"something\"}`\n      - `xmlns:ns=\"http://myns.com/ns\"` parses to `map[\"xmlns:ns\"]interface{}{\"http://myns.com/ns\"}`\n\nBoth\n\n   - By default, \"Nan\", \"Inf\", and \"-Inf\" values are not cast to float64.  If you want them\n     to be cast, set a flag to cast them  using CastNanInf(true).\n\n\u003ch4\u003eXML encoding conventions\u003c/h4\u003e\n\n   - 'nil' `Map` values, which may represent 'null' JSON values, are encoded as `\u003ctag/\u003e`.\n     NOTE: the operation is not symmetric as `\u003ctag/\u003e` elements are decoded as `tag:\"\"` `Map` values,\n           which, then, encode in JSON as `\"tag\":\"\"` values.\n   - ALSO: there is no guarantee that the encoded XML doc will be the same as the decoded one.  (Go\n           randomizes the walk through map[string]interface{} values.) If you plan to re-encode the\n           Map value to XML and want the same sequencing of elements look at NewMapXmlSeq() and\n           mv.XmlSeq() - these try to preserve the element sequencing but with added complexity when\n           working with the Map representation.\n\n\u003ch4\u003eRunning \"go test\"\u003c/h4\u003e\n\nBecause there are no guarantees on the sequence map elements are retrieved, the tests have been \nwritten for visual verification in most cases.  One advantage is that you can easily use the \noutput from running \"go test\" as examples of calling the various functions and methods.\n\n\u003ch4\u003eMotivation\u003c/h4\u003e\n\nI make extensive use of JSON for messaging and typically unmarshal the messages into\n`map[string]interface{}` values.  This is easily done using `json.Unmarshal` from the\nstandard Go libraries.  Unfortunately, many legacy solutions use structured\nXML messages; in those environments the applications would have to be refactored to\ninteroperate with my components.\n\nThe better solution is to just provide an alternative HTTP handler that receives\nXML messages and parses it into a `map[string]interface{}` value and then reuse\nall the JSON-based code.  The Go `xml.Unmarshal()` function does not provide the same\noption of unmarshaling XML messages into `map[string]interface{}` values. So I wrote\na couple of small functions to fill this gap and released them as the x2j package.\n\nOver the next year and a half additional features were added, and the companion j2x\npackage was released to address XML encoding of arbitrary JSON and `map[string]interface{}`\nvalues.  As part of a refactoring of our production system and looking at how we had been\nusing the x2j and j2x packages we found that we rarely performed direct XML-to-JSON or\nJSON-to_XML conversion and that working with the XML or JSON as `map[string]interface{}`\nvalues was the primary value.  Thus, everything was refactored into the mxj package.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclbanning%2Fmxj","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclbanning%2Fmxj","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclbanning%2Fmxj/lists"}