{"id":22096010,"url":"https://github.com/perimeterx/marshmallow","last_synced_at":"2025-05-15T04:08:10.199Z","repository":{"id":38323513,"uuid":"481129935","full_name":"PerimeterX/marshmallow","owner":"PerimeterX","description":"Marshmallow provides a flexible and performant JSON unmarshalling in Go. It specializes in dealing with unstructured struct - when some fields are known and some aren't, with zero performance overhead nor extra coding needed.","archived":false,"fork":false,"pushed_at":"2023-07-03T07:05:49.000Z","size":4149,"stargazers_count":384,"open_issues_count":1,"forks_count":11,"subscribers_count":15,"default_branch":"main","last_synced_at":"2025-05-15T04:08:00.233Z","etag":null,"topics":["go","golang","json","performance"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PerimeterX.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-04-13T08:16:03.000Z","updated_at":"2025-04-14T09:49:50.000Z","dependencies_parsed_at":"2024-01-08T15:34:40.235Z","dependency_job_id":null,"html_url":"https://github.com/PerimeterX/marshmallow","commit_stats":{"total_commits":21,"total_committers":4,"mean_commits":5.25,"dds":"0.19047619047619047","last_synced_commit":"6f0f9e39c2ce0a85fe4e3f0bd636fc0bb5d1ec2b"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PerimeterX%2Fmarshmallow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PerimeterX%2Fmarshmallow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PerimeterX%2Fmarshmallow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PerimeterX%2Fmarshmallow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PerimeterX","download_url":"https://codeload.github.com/PerimeterX/marshmallow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254270656,"owners_count":22042860,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","golang","json","performance"],"created_at":"2024-12-01T04:09:21.951Z","updated_at":"2025-05-15T04:08:05.148Z","avatar_url":"https://github.com/PerimeterX.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Marshmallow\n\n![Marshmallow Campfire](https://raw.githubusercontent.com/PerimeterX/marshmallow/assets/campfire.png)\n\n[![CodeQL Status](https://img.shields.io/github/actions/workflow/status/perimeterx/marshmallow/codeql.yml?branch=main\u0026logo=github\u0026label=CodeQL)](https://github.com/PerimeterX/marshmallow/actions/workflows/codeql.yml?query=branch%3Amain++)\n[![Run Tests](https://img.shields.io/github/actions/workflow/status/perimeterx/marshmallow/go.yml?branch=main\u0026logo=github\u0026label=Run%20Tests)](https://github.com/PerimeterX/marshmallow/actions/workflows/go.yml?query=branch%3Amain)\n[![Dependency Review](https://img.shields.io/github/actions/workflow/status/perimeterx/marshmallow/dependency-review.yml?logo=github\u0026label=Dependency%20Review)](https://github.com/PerimeterX/marshmallow/actions/workflows/dependency-review.yml?query=branch%3Amain)\n[![Go Report Card](https://goreportcard.com/badge/github.com/perimeterx/marshmallow)](https://goreportcard.com/report/github.com/perimeterx/marshmallow)\n![Manual Code Coverage](https://img.shields.io/badge/coverage-92.6%25-green)\n[![Go Reference](https://pkg.go.dev/badge/github.com/perimeterx/marshmallow.svg)](https://pkg.go.dev/github.com/perimeterx/marshmallow)\n[![Licence](https://img.shields.io/github/license/perimeterx/marshmallow)](LICENSE)\n[![Latest Release](https://img.shields.io/github/v/release/perimeterx/marshmallow)](https://github.com/PerimeterX/marshmallow/releases)\n![Top Languages](https://img.shields.io/github/languages/top/perimeterx/marshmallow)\n[![Issues](https://img.shields.io/github/issues-closed/perimeterx/marshmallow?color=%238250df\u0026logo=github)](https://github.com/PerimeterX/marshmallow/issues)\n[![Pull Requests](https://img.shields.io/github/issues-pr-closed-raw/perimeterx/marshmallow?color=%238250df\u0026label=merged%20pull%20requests\u0026logo=github)](https://github.com/PerimeterX/marshmallow/pulls)\n[![Commits](https://img.shields.io/github/last-commit/perimeterx/marshmallow)](https://github.com/PerimeterX/marshmallow/commits/main)\n[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.1-4baaaa.svg)](CODE_OF_CONDUCT.md)\n\n\u003cimg align=\"right\" width=\"200\" alt=\"marshmallow-gopher\" src=\"https://raw.githubusercontent.com/PerimeterX/marshmallow/assets/sticker7.png\"\u003e\n\nMarshmallow package provides a simple API to perform flexible and performant JSON unmarshalling in Go.\n\nMarshmallow specializes in dealing with **unstructured struct** - when some fields are known and some aren't,\nwith zero performance overhead nor extra coding needed.\nWhile unmarshalling, marshmallow allows fully retaining the original data and access\nit via a typed struct and a dynamic map.\n\n## Contents\n\n- [Install](#install)\n- [Usage](#usage)\n- [Performance Benchmark And Alternatives](#performance-benchmark-and-alternatives)\n- [When Should I Use Marshmallow](#when-should-i-use-marshmallow)\n- [API](#api)\n\n## Install\n\n```sh\ngo get -u github.com/perimeterx/marshmallow\n```\n\n## Usage\n\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"github.com/perimeterx/marshmallow\"\n)\n\nfunc main() {\n\tv := struct {\n\t\tFoo string `json:\"foo\"`\n\t\tBoo []int  `json:\"boo\"`\n\t}{}\n\tresult, err := marshmallow.Unmarshal([]byte(`{\"foo\":\"bar\",\"boo\":[1,2,3],\"goo\":12.6}`), \u0026v)\n\tfmt.Printf(\"v=%+v, result=%+v, err=%v\", v, result, err)\n\t// Output: v={Foo:bar Boo:[1 2 3]}, result=map[boo:[1 2 3] foo:bar goo:12.6], err=\u003cnil\u003e\n}\n```\n\n**Examples can be found [here](example_test.go)**\n\n## Performance Benchmark And Alternatives\n\nMarshmallow performs best when dealing with mixed data - when some fields are known and some are unknown.\nMore info [below](#when-should-i-use-marshmallow).\nOther solutions are available for this kind of use case, each solution is explained and documented in the link below.\nThe full benchmark test can be found\n[here](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go).\n\n|Benchmark|Iterations|Time/Iteration|Bytes Allocated|Allocations|\n|--|--|--|--|--|\n|[unmarshall twice](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L40)|228693|5164 ns/op|1640 B/op|51 allocs/op|\n|[raw map](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L66)|232236|5116 ns/op|2296 B/op|53 allocs/op|\n|[go codec](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L121)|388442|3077 ns/op|2512 B/op|37 allocs/op|\n|[marshmallow](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L16)|626168|1853 ns/op|608 B/op|18 allocs/op|\n|[marshmallow without populating struct](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L162)|678616|1751 ns/op|608 B/op|18 allocs/op|\n\n![marshmallow performance comparison](https://raw.githubusercontent.com/PerimeterX/marshmallow/e45088ca20d4ea5be4143d418d12da63a68d6dfd/performance-chart.svg)\n\n**Marshmallow provides the best performance (up to X3 faster) while not requiring any extra coding.**\nIn fact, marshmallow performs as fast as normal `json.Unmarshal` call, however, such a call causes loss of data for all\nthe fields that did not match the given struct. With marshmallow you never lose any data.\n\n|Benchmark|Iterations|Time/Iteration|Bytes Allocated|Allocations|\n|--|--|--|--|--|\n|[marshmallow](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L16)|626168|1853 ns/op|608 B/op|18 allocs/op|\n|[native library](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L143)|652106|1845 ns/op|304 B/op|11 allocs/op|\n|[marshmallow without populating struct](https://github.com/PerimeterX/marshmallow/blob/8c5bba9e6dc0033f4324eca554737089a99f6e5e/benchmark_test.go#L162)|678616|1751 ns/op|608 B/op|18 allocs/op|\n\n## When Should I Use Marshmallow\n\nMarshmallow is best suited for use cases where you are interested in all the input data, but you have predetermined\ninformation only about a subset of it. For instance, if you plan to reference two specific fields from the data, then\niterate all the data and apply some generic logic. How does it look with the native library:\n\n```go\nfunc isAllowedToDrive(data []byte) (bool, error) {\n\tresult := make(map[string]interface{})\n\terr := json.Unmarshal(data, \u0026result)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tage, ok := result[\"age\"]\n\tif !ok {\n\t\treturn false, nil\n\t}\n\ta, ok := age.(float64)\n\tif !ok {\n\t\treturn false, nil\n\t}\n\tif a \u003c 17 {\n\t\treturn false, nil\n\t}\n\n\thasDriversLicense, ok := result[\"has_drivers_license\"]\n\tif !ok {\n\t\treturn false, nil\n\t}\n\th, ok := hasDriversLicense.(bool)\n\tif !ok {\n\t\treturn false, nil\n\t}\n\tif !h {\n\t\treturn false, nil\n\t}\n\n\tfor key := range result {\n\t\tif strings.Contains(key, \"prior_conviction\") {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\treturn true, nil\n}\n```\n\nAnd with marshmallow:\n\n```go\nfunc isAllowedToDrive(data []byte) (bool, error) {\n\tv := struct {\n\t\tAge               int  `json:\"age\"`\n\t\tHasDriversLicense bool `json:\"has_drivers_license\"`\n\t}{}\n\tresult, err := marshmallow.Unmarshal(data, \u0026v)\n\tif err != nil {\n\t\treturn false, err\n\t}\n\n\tif v.Age \u003c 17 || !v.HasDriversLicense {\n\t\treturn false, nil\n\t}\n\n\tfor key := range result {\n\t\tif strings.Contains(key, \"prior_conviction\") {\n\t\t\treturn false, nil\n\t\t}\n\t}\n\n\treturn true, nil\n}\n```\n\n## API\n\nMarshmallow exposes two main API functions -\n[Unmarshal](https://github.com/PerimeterX/marshmallow/blob/0e0218ab860be8a4b5f57f5ff239f281c250c5da/unmarshal.go#L27)\nand\n[UnmarshalFromJSONMap](https://github.com/PerimeterX/marshmallow/blob/0e0218ab860be8a4b5f57f5ff239f281c250c5da/unmarshal_from_json_map.go#L37).\nWhile unmarshalling, marshmallow supports the following optional options:\n\n* Setting the mode for handling invalid data using the [WithMode](https://github.com/PerimeterX/marshmallow/blob/0e0218ab860be8a4b5f57f5ff239f281c250c5da/options.go#L30) function.\n* Excluding known fields from the result map using the [WithExcludeKnownFieldsFromMap](https://github.com/PerimeterX/marshmallow/blob/457669ae9973895584f2636eabfc104140d3b700/options.go#L50) function. \n* Skipping struct population to boost performance using the [WithSkipPopulateStruct](https://github.com/PerimeterX/marshmallow/blob/0e0218ab860be8a4b5f57f5ff239f281c250c5da/options.go#L41) function.\n\nIn order to capture unknown nested fields, structs must implement [JSONDataErrorHandler](https://github.com/PerimeterX/marshmallow/blob/195c994aa6e3e0852601ad9cf65bcddef0dd7479/options.go#L76).\nMore info [here](https://github.com/PerimeterX/marshmallow/issues/15). \n\nMarshmallow also supports caching of refection information using \n[EnableCache](https://github.com/PerimeterX/marshmallow/blob/d3500aa5b0f330942b178b155da933c035dd3906/cache.go#L40)\nand\n[EnableCustomCache](https://github.com/PerimeterX/marshmallow/blob/d3500aa5b0f330942b178b155da933c035dd3906/cache.go#L35).\n\n## Contact and Contribute\n\nReporting issues and requesting features may be done in our [GitHub issues page](https://github.com/PerimeterX/marshmallow/issues).\nDiscussions may be conducted in our [GitHub discussions page](https://github.com/PerimeterX/marshmallow/discussions).\nFor any further questions or comments you can reach us out at [open-source@humansecurity.com](mailto:open-source@humansecurity.com).\n\nAny type of contribution is warmly welcome and appreciated ❤️\nPlease read our [contribution](CONTRIBUTING.md) guide for more info.\n\nIf you're looking for something to get started with, tou can always follow our [issues page](https://github.com/PerimeterX/marshmallow/issues) and look for\n[good first issue](https://github.com/PerimeterX/marshmallow/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22) and\n[help wanted](https://github.com/PerimeterX/marshmallow/issues?q=is%3Aissue+label%3A%22help+wanted%22+is%3Aopen) labels.\n\n## Marshmallow Logo\n\nMarshmallow logo and assets by [Adva Rom](https://www.linkedin.com/in/adva-rom-7a6738127/) are licensed under a \u003ca rel=\"license\" href=\"http://creativecommons.org/licenses/by/4.0/\"\u003eCreative Commons Attribution 4.0 International License\u003c/a\u003e.\u003cbr /\u003e\n\n![Marshmallow Logo](https://raw.githubusercontent.com/PerimeterX/marshmallow/assets/marshmallow.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperimeterx%2Fmarshmallow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fperimeterx%2Fmarshmallow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fperimeterx%2Fmarshmallow/lists"}