{"id":27204600,"url":"https://github.com/karaatanassov/go_polymorphic_json","last_synced_at":"2025-04-09T22:58:47.276Z","repository":{"id":60115943,"uuid":"342198095","full_name":"karaatanassov/go_polymorphic_json","owner":"karaatanassov","description":"Demonstration of using polymorphic JSON types with golang.","archived":false,"fork":false,"pushed_at":"2023-01-10T23:43:01.000Z","size":37,"stargazers_count":17,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-09T22:58:43.634Z","etag":null,"topics":["discriminator","go","golang","json","openapi","polymorphism"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/karaatanassov.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-02-25T09:56:01.000Z","updated_at":"2024-12-03T15:23:11.000Z","dependencies_parsed_at":"2023-02-08T20:47:11.384Z","dependency_job_id":null,"html_url":"https://github.com/karaatanassov/go_polymorphic_json","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karaatanassov%2Fgo_polymorphic_json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karaatanassov%2Fgo_polymorphic_json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karaatanassov%2Fgo_polymorphic_json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/karaatanassov%2Fgo_polymorphic_json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/karaatanassov","download_url":"https://codeload.github.com/karaatanassov/go_polymorphic_json/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248125608,"owners_count":21051768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["discriminator","go","golang","json","openapi","polymorphism"],"created_at":"2025-04-09T22:58:46.757Z","updated_at":"2025-04-09T22:58:47.261Z","avatar_url":"https://github.com/karaatanassov.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Polymorphic JSON with Go\n\nIn this article and examples we look at the problem of handling polymorphic\nJSON structures in the Golang programming language.\n\nDeal with polymorphic serialized JSON data is a problem discussed in the Go\ncommunity. Polymorhpisn can be found in existing APIs and is sometimes needed\nfor new APIs. Unfortunately the default Go JSON code does not handle\npolymorphism out of the box and hence the motivation for this work.\n\nLet us first understand what polymorphism is and how it maps to Go language.\n\nAs per Wikipedia [Polymorphism](https://en.wikipedia.org/wiki/Polymorphism_(computer_science))\nis\n\n\u003e In programming languages and type theory, polymorphism is the provision of a\n\u003e single interface to entities of different types\n\nIn the JSON serialization format Polymorphism is used to select among one of\nmultiple data structure types. Those data types may be a flat enumeration or\nrepresent a  hierarchical system such as the classification of living things or\nexception hierarchy.\n\nIn this work we would detail the more complex case of deep hierarchical\nstructure. The same approach is applicable to a flat enumeration of data\nstructure types.\n\nOne example of polymorphic JSON API is [RFC 7946 Geo JSON](https://tools.ietf.org/html/rfc7946).\n```json\n{\n    \"type\": \"Point\",\n    \"coordinates\": [102.0, 0.5]\n}\n// OR\n{\n    \"type\": \"Polygon\",\n    \"coordinates\": [\n        [\n            [100.0, 0.0],\n            [101.0, 0.0],\n            [101.0, 1.0],\n            [100.0, 1.0],\n            [100.0, 0.0]\n        ]\n    ]\n}\n```\nIn this specification we would look into a simple error model that would be a\ncommon chore for API authors and consumers.\n```json\n{\n    \"errors\": [\n        {\n            \"Kind\" : \"Fault\",\n            \"Message\": \"Something went wrong.\",\n            \"Cause\": {  \"Kind\": \"Fault\", \"Message\": \"Missing file\" }\n        },{\n            \"Kind\" : \"RuntimeFault\",\n            \"Message\": \"Unexpected error\"\n        },{\n            \"Kind\" : \"Not Found\",\n            \"Message\": \"The cat Lucie is missing.\",\n            \"Obj\": \"Lucie\",\n            \"ObjKind\": \"Cat\"\n        }\n    ]\n}\n```\n## Data Model in Go\nThe first task is to define the data objects in Go. This is rather\nstraightforward job:\n```go\ntype FaultStruct struct {\n\tMessage string\n\tCause   *Fault\n}\ntype RuntimeFaultStruct struct {\n\tFault\n}\ntype NotFoundStruct struct {\n\tRuntimeFault\n\tObjKind string\n\tObj     string\n}\n```\nNote how embedding can be used to construct more complex types without repeating\nthe definitions. This is similar to the `allOf` construct in JSON schema.\n\nWe have now defined a schema of three objects that extend each other. However we\ndo not get Polymorphic behavior. In other words [the following results in error](https://play.golang.org/p/q55Z6zdA63Y)\n```go\nfunc printFault(f *Fault) {\n\tfmt.Println(\"The fault message is:\", f.Message);\n}\n\nfunc main() {\n\tprintFault(\u0026RuntimeFault{ Fault { Message: \"test\" } })\n}\n```\nWe cannot use `RuntimeFault` or `NotFound` in place where `Fault` is needed.\nLet's see how to solve this.\n\n## Polymorphism in Go\n\nThe Go language provides us with the concept of [interfaces](https://golang.org/doc/effective_go#interfaces})\nimplemented through methods on various types. It is with interfaces that\ndifferent data structures can be used interchangably and exhibit polymorphic\nbehavior.\n\nWe can apply interfaces in several ways to our example to get the desired\npolymorphic behavour. The first way we look at involves getters and setters.\nThis is simpler to understand while not idiomatic for Go. The second apporach\ndefines interfaces that provide access to the underlying data structure. It is\nnot as obvious yet produces cleaner and more concise go code.\n\n### Using Interfaces with Accessors\n\nApplying interfaces to our example above produces [working code](https://play.golang.org/p/UckCCDz4wYT)\n```go\ntype FaultInterface interface {\n\tGetMessage() string\n}\n\nfunc (f *Fault) GetMessage() string {\n\treturn f.Message\n}\nfunc printFault(f FaultInterface) {\n\tfmt.Println(\"The fault message is:\", f.GetMessage())\n}\nfunc main() {\n\tprintFault(\u0026RuntimeFault{Fault{Message: \"test\"}})\n}\n```\nIn similar way an interface can be used as return type, member of another\nstructure or element type of a slice.\n\nIn addition to data model we would need to define interfaces for each type to have polymorphic behavior.\n\nI prefer to keep things tidy and kept the models and their implementations in\nthe `models` package. While the interface definitions reside in the `interfaces`\npackage. Thus we get the following definitions:\n\n```go\ntype Fault interface {\n\tGetMessage() string\n\tSetMessage(string)\n\tGetCause() Fault\n\tSetCause(Fault)\n}\ntype RuntimeFault interface {\n\tFault\n}\ntype NotFound interface {\n\tRuntimeFault\n\tGetObjKind() string\n\tSetObjKind(string)\n\tGetObj() string\n\tSetObj(string)\n}\n```\nThe code is still [DRY](https://en.wikipedia.org/wiki/Don%27t_repeat_yourself).\nThere is no need to include members of generic abstractions into\nspecialized ones. Instead Go provides us with embedding to reuse existing code.\n\nOf course we will need to implement the methods in the `models` package at\nleast once on the most generic type they appear on. We could also implement some\nmethods more than once on more specific types if there is specific business\nlogic to include at that level of abstraction.\n\n### No Accessors\n\n\n\n## Rendering JSON\n\nRendering JSON from Go structure even referred through an interface is a\nstraightforward task. In our case we want to add the class name or discriminator\nin the JSON payload for the recipients to know what type of fault they are\nreceiving. The following simple pattern works in Go:\n```go\nfunc (nfo *NotFound) MarshalJSON() ([]byte, error) {\n\ttype marshalable NotFound\n\treturn json.Marshal(struct {\n\t\tKind string\n\t\tmarshalable\n\t}{\n\t\tKind:        \"NotFound\",\n\t\tmarshalable: marshalable(*nfo),\n\t})\n}\n```\nHere is what this method does:\n\n1. It declares a method for marshalling `NotFound` objects. Go discovers it\ndynamically by checking if the object implements `encoding/json.Unmarshaler`.\n1. A new type is declared that has no marshaling logic - `marshalable` this is\nneeded to evade recursion\n1. An anonymous struct type is declared that provides  additional field for the\ndiscriminator.\n1. An object of the new type is created that embeds the current `NotFound`\ninstance data and adds the `NotFound` value for the `Kind` field.\n\nIt is good idea to have the field `Kind` on top. This way Go will render it\nfirst on the wire and clients will consume the output more efficiently.\n\nAfter we add these methods to our error types the serialization starts to work.\n\n```go\nfunc main() {\n\tvar fi FaultInterface = \u0026RuntimeFault{Fault{Message: \"test\"}}\n\tdata, err := json.Marshal(fi)\n\tif err != nil {\n\t\tfmt.Println(\"Cannot write JSON\", err)\n\t\treturn\n\t}\n\tfmt.Println(\"JSON:\", string(data))\n}\n```\nprints:\n```json\n{\"Kind\":\"RuntimeFault\",\"Message\":\"test\",\"Cause\":null}\n```\nHere is the [evolving example](https://play.golang.org/p/1A0uMnBfcYV).\n\n\n## Reading JSON\n\nReading back the JSON values is slightly more convoluted. The biggest obstacle\nis that Go has no easy way to associate functionality to an interface type. Thus\nit is not possible to implement unmarshal method on our interface types. Instead\nwe need to take care of unmarshaling interface one layer above them. There are\nseveral scenarios where we can see the need to unmarshal interfaces:\n\n1. Read an top level JSON object into an interface\n2. Read member field of an object into an interface\n3. Read array member into an interface\n\nReading a JSON document like the one we have above into na interface is simplest.\nWe need a function that receives a `[]byte` and returns `interface.Fault`. Let\nus call this `UnmarshalFault`:\n\n```go\nfunc UnmarshalFault(in []byte) (Fault, error) {\n\td := \u0026struct {\n\t\tKind string\n\t}{}\n\t// Double pointer detects null values\n\terr := json.Unmarshal(in, \u0026d)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif d == nil {\n\t\treturn nil, nil\n\t}\n\tkind := d.Kind\n\n\tvar res Fault\n\tswitch kind {\n\tcase \"NotFound\":\n\t\tres = \u0026NotFound{}\n\tcase \"RuntimeFault\":\n\t\tres = \u0026RuntimeFault{}\n\tdefault: // Error on default or try to use base type?\n\t\tres = \u0026Fault{}\n\t}\n\tjson.Unmarshal(in, res)\n\n\treturn res, nil\n}\n```\nThe basic operation of the function is as follows:\n\n1. Scan the incoming JSON bytes for the discriminator field `Kind`\n2. Depending on the type the proper struct type is initialized and read from the\nwire\n\nThis pretty much does the trick for basic interface unmarshaling.\n\nHere is the [sample getting bigger](https://play.golang.org/p/6k2nFYd-dAN)\n\n### Reading JSON fields\n\nThe last challenge is reading interfaces when they are members of an object or\nan array.\n\nTo achieve this each object that has field of polymorphic type will need custom unmarshaling logic that reads into a type with placeholder fields that can be\nunmarshaled into temporary object and later converted into the proper type.\n\nFor [example](models/field_test.go)\n\n```go\nvar _ json.Unmarshaler = \u0026Container{}\n\nfunc (c *Container) UnmarshalJSON(in []byte) error {\n\t// Deserialize into temp object\n\ttemp := struct {\n\t\tFaultField json.RawMessage\n\t}{}\n\terr := json.Unmarshal(in, \u0026temp)\n\tif err != nil {\n\t\treturn err\n\t}\n\tc.FaultField = nil\n\tif temp.FaultField != nil {\n\t\tc.FaultField, err = UnmarshalFault(temp.FaultField)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\treturn nil\n}\n```\nOur goal in the code above is to unmarshal `Container` structure. To achieve it\nwe first unmarshal into  spacial structure that has `json.RawMessage` field\ninstead of the `Fault` interface type. We use the `UnmarshalFault` function to\nconvert the `json.RawMessage` to `Fault`,\n\nIn this way we can now easily read our `Container` object from a JSON file and\nget the correct `Fault` instance.\n\n```go\n\tc := Container{}\n\terr = json.Unmarshal(b, \u0026c)\n```\n\n### Reading JSON arrays\n\nWorking with arrays is similar to object fields. We need to first read into\nspecial slice of `json.RawMEssage` elements and then build the proper slice of\n`Fault` interface implementations.\n\nYou can see the code in [array_test.go](models/array_test.go)\n\n```go\ntype ArrayContainer struct {\n\tFaults []Fault\n}\n\nvar _ json.Unmarshaler = \u0026ArrayContainer{}\n\nfunc (c *ArrayContainer) UnmarshalJSON(in []byte) error {\n\t// Deserialize into temp object of utility class\n\ttemp := struct {\n\t\tFaults []json.RawMessage\n\t}{}\n\terr := json.Unmarshal(in, \u0026temp)\n\tif err != nil {\n\t\treturn err\n\t}\n\tc.Faults = nil\n\tif temp.Faults != nil {\n\t\tc.Faults = []Fault{}\n\t\tfor _, rawFault := range temp.Faults {\n\t\t\tif rawFault == nil {\n\t\t\t\tc.Faults = append(c.Faults, nil)\n\t\t\t}\n\t\t\tfault, err := UnmarshalFault(rawFault)\n\t\t\tif err != nil {\n\t\t\t\treturn err\n\t\t\t}\n\t\t\tc.Faults = append(c.Faults, fault)\n\t\t}\n\t}\n\treturn nil\n}\n```\n\n### What about the Fault cause\n\nThe `Fault` object contains a `Cause` field that links to a related `Fault` object. We need to change the field from pointer to struct type to interface as to allow polymorphic behavior. Further we need to add `UnmarshalJSON` operations to all types in the hierarchy as to correctly deserialize. The methods on every type need to care about all fields and cannot delegate to the embedded types.\n\n```go\nfunc (nfo *NotFound) UnmarshalJSON(in []byte) error {\n\tpxy := \u0026struct {\n\t\tMessage string\n\t\tCause   json.RawMessage\n\t\tObjKind string\n\t\tObj     string\n\t}{}\n\terr := json.Unmarshal(in, pxy)\n\tif err != nil {\n\t\treturn err\n\t}\n\tvar cause Fault\n\tif pxy.Cause != nil {\n\t\tcause, err = UnmarshalFault(pxy.Cause)\n\t\tif err != nil {\n\t\t\treturn err\n\t\t}\n\t}\n\tnfo.Message = pxy.Message\n\tnfo.Cause = cause\n\tnfo.Obj = pxy.Obj\n\tnfo.ObjKind = pxy.ObjKind\n\treturn nil\n}\n```\n\n### How about type safety\n\nUsing deep hierarchies in Go may be problematic as Go interface conversions are based on the presence methods. In our example any `Fault` object works well as `RuntimeFault`. This is different from the behavior of C++ and Java where objects with no members are used to provide type safety and classification. This functionality can be emulated by adding synthetic member functions for each interface type in a hierarchy. For example the following prevents a `Fault` to be converted to `RuntimeFault`\n\n```go\ntype RuntimeFault interface {\n\tFault\n    ZzRuntimeFault()\n}\n\n// ZzRuntimeFault is a marker\nfunc (rf *RuntimeFault) ZzRuntimeFault() {\n}\n```\n\n### How is unmarshaling working in the hierarchy?\n\n `RuntimeFault` fields are polymorphic and are not root of a hierarchy. One option is to have `UnmarshalRuntimeFault` bank on the root objects like `Fault` and invoke `UnmarshalFault` function.  This will save code size as in a large hierarchy the unmashal methods may become too big.\n\n```go\n func UnmarshalRuntimeFault(in []byte) (RuntimeFault, error) {\n\tfault, err := UnmarshalFault(in)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tif runtimeFault, ok := fault.(RuntimeFault); ok {\n\t\treturn runtimeFault, nil\n\t}\n\treturn nil, fmt.Errorf(\"Cannot unmarshal RuntimeFault %v\", fault)\n}\n```\n\n## Conclusion and next steps\n\nThis article and sample code illustrate the basic handling of polymorphic JSON in Go. We see that out of the box support is lacking. Yet a little bit of creativity helps us get near native experience with polymorphic unmarshaling in Go.\n\nThe article and sample code leave out some details.\n\nOne area to discuss is how this work can be mapped onto polymorphic OpenAPI schema. The combination of `allOf` and `discriminator` constructs used in OpenAPI generator with Java provides good base.\n\nThe performance of the switch statement in `UnmarshalFault` on a string when thousands of classes exist in a hierarchy may require optimized implementation. For example use of state machine that iterates the characters to discern different options and assert valid sequences.\n\n## References\n\nThis article builds on a number of previous implementations. Some of those are\nlisted below:\n\n* [StackOverflow: Polymorphic JSON unmarshalling of embedded structs](https://stackoverflow.com/questions/44380095/polymorphic-json-unmarshalling-of-embedded-structs)\n* [unmarshal_interface.go](https://gist.github.com/tkrajina/aec8d1b15b088c20f0df4afcd5f0c511)\n* [go-swagger](https://github.com/go-swagger/go-swagger)\n* [JSON polymorphism in Go](https://alexkappa.medium.com/json-polymorphism-in-go-4cade1e58ed1)\n* [StackOverflow: Can I use MarshalJSON to add arbitrary fields to a json encoding in golang?](https://stackoverflow.com/questions/23045884/can-i-use-marshaljson-to-add-arbitrary-fields-to-a-json-encoding-in-golang)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaraatanassov%2Fgo_polymorphic_json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaraatanassov%2Fgo_polymorphic_json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaraatanassov%2Fgo_polymorphic_json/lists"}