{"id":18000185,"url":"https://github.com/migamake/json-autotype","last_synced_at":"2025-05-12T14:29:37.136Z","repository":{"id":16584709,"uuid":"19338956","full_name":"migamake/json-autotype","owner":"migamake","description":"Automatic Haskell type inference from JSON input","archived":false,"fork":false,"pushed_at":"2023-09-21T00:08:59.000Z","size":924,"stargazers_count":149,"open_issues_count":13,"forks_count":23,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-12T14:29:30.022Z","etag":null,"topics":["elm","hackage","hacktoberfest2019","haskell","inference","json","json-autotype","parse","parser","unification","union"],"latest_commit_sha":null,"homepage":null,"language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/migamake.png","metadata":{"files":{"readme":"README.md","changelog":"changelog.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-05-01T07:52:05.000Z","updated_at":"2025-01-07T16:06:30.000Z","dependencies_parsed_at":"2024-06-21T03:54:27.446Z","dependency_job_id":"d5293336-69f1-45fd-a7d7-bcb83e46cc9f","html_url":"https://github.com/migamake/json-autotype","commit_stats":{"total_commits":566,"total_committers":10,"mean_commits":56.6,"dds":0.5459363957597173,"last_synced_commit":"3b03c49c323eea32fa16f625ce54db1f9bfa155c"},"previous_names":["mgajda/json-autotype"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/migamake%2Fjson-autotype","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/migamake%2Fjson-autotype/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/migamake%2Fjson-autotype/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/migamake%2Fjson-autotype/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/migamake","download_url":"https://codeload.github.com/migamake/json-autotype/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253754756,"owners_count":21958903,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elm","hackage","hacktoberfest2019","haskell","inference","json","json-autotype","parse","parser","unification","union"],"created_at":"2024-10-29T23:10:16.208Z","updated_at":"2025-05-12T14:29:37.113Z","avatar_url":"https://github.com/migamake.png","language":"Haskell","funding_links":["https://paypal.me/MichalJan"],"categories":[],"sub_categories":[],"readme":"json-autotype\n=============\nTakes a JSON format input, and generates automatic Haskell type declarations.\n\nParser and printer instances are derived using [Aeson](http://hackage.haskell.org/package/aeson).\n\nThe program uses union type unification to trim output declarations. The types of same attribute tag and similar attribute set, are automatically unified using recognition by attribute set matching. (This option can be optionally turned off, or a set of unified types may be given explicitly.) `:|:` alternatives (similar to `Either`) are used to assure that all `JSON` inputs seen in example input file are handled correctly.\n\nI should probably write a short paper to explain the methodology.\n\n[![Release build](https://gitlab.com/migamake/json-autotype/badges/master/pipeline.svg)](https://gitlab.com/migamake/json-autotype/commits/master)\n[![Hackage](https://img.shields.io/hackage/v/json-autotype.svg)](https://hackage.haskell.org/package/json-autotype)\n[![Hackage Dependencies](https://img.shields.io/hackage-deps/v/json-autotype.svg?style=flat)](http://packdeps.haskellers.com/feed?needle=json-autotype)\n[![Docker Automated build](https://img.shields.io/docker/automated/migamake/json-autotype.svg)](https://hub.docker.com/r/migamake/json-autotype/)\n[![Docker image size](https://img.shields.io/microbadger/image-size/migamake/json-autotype.svg)](https://hub.docker.com/r/migamake/json-autotype/)\n\nDetails on official releases are on [Hackage](https://hackage.haskell.org/package/json-autotype)\nWe currently support code generation to [Haskell](https://www.haskell.org), and [Elm](https://elm-lang.org).\n\n_Please [volunteer help](https://gitter.im/dataHaskell/json-autotype) or [provide financial support](https://paypal.me/MichalJan), if you want your favourite language supported too!_\nExpression of interest in particular feature may be filed as [GitHub issue](https://github.com/mgajda/json-autotype/issues/new).\n\n\nUSAGE:\n======\nAfter installing with `cabal install json-autotype`, you might generate stub code for the parser:\n\n```\n    json-autotype input1.json ... inputN.json -o MyFormat.hs\n```\n\nThen you might test the parser by running it on an input file:\n\n```\n    runghc MyFormat.hs input.json\n```\n\nAt this point you may see data structure generated automatically for you.\nThe more input files you give to the inference engine `json-autotype`,\nthe more precise type description will be.\n\nAlgorithm will also suggest which types look similar, based on a set of attribute names,\nand unify them unless specifically instructed otherwise.\n\nThe goal of this program is to make it easy for users of big JSON APIs to generate entries from\nexample data.\n\nOccasionally you might find a valid JSON for which `json-autotype` doesn't generate a correct parser.\nYou may either edit the resulting file _and_ send it to the author as a test case for future release.\n\nPatches and suggestions are welcome.\n\nYou can run [with Docker](https://hub.docker.com/r/migamake/json-autotype/):\n```\ndocker run -it migamake/json-autotype\n```\n\nEXAMPLES:\n=========\n\nThe most simple example:\n```\n    {\n        \"colorsArray\":[{\n                \"colorName\":\"red\",\n                \"hexValue\":\"#f00\"\n            },\n            {\n                \"colorName\":\"green\",\n                \"hexValue\":\"#0f0\"\n            },\n            {\n                \"colorName\":\"blue\",\n                \"hexValue\":\"#00f\"\n            }\n        ]\n    }\n```\n\nIt will produce the module with the following datatypes and TH calls for JSON parser derivations:\n```\n    data ColorsArray = ColorsArray {\n        colorsArrayHexValue    :: Text,\n        colorsArrayColorName :: Text\n      } deriving (Show,Eq)\n\n    data TopLevel = TopLevel {\n        topLevelColorsArray :: ColorsArray\n      } deriving (Show,Eq)\n```\nNote that attribute names match the names of JSON dictionary keys.\n\nAnother example with ambiguous types:\n```\n    {\n        \"parameter\":[{\n                \"parameterName\":\"apiVersion\",\n                \"parameterValue\":1\n            },\n            {\n                \"parameterName\":\"failOnWarnings\",\n                \"parameterValue\":false\n            },\n            {\n                \"parameterName\":\"caller\",\n                \"parameterValue\":\"site API\"\n            }]\n    }\n```\nIt will produce quite intuitive result (plus extra parentheses, and class derivations):\n\n```\n    data Parameter = Parameter {\n        parameterParameterValue :: Bool :|: Int :|: Text,\n        parameterParameterName :: Text\n      }\n\n    data TopLevel = TopLevel {\n        topLevelParameter :: Parameter\n      }\n```\n\nReal-world use case examples are provided in the package [source repository](https://github.com/mgajda/json-autotype/tree/master/test).\n\nMethodology:\n============\n1. JSON-Autotype uses its own [union type system](https://github.com/mgajda/json-autotype/blob/master/Data/Aeson/AutoType/Type.hs) to derive types from JSON documents as the first step.\n2. Then it finds all those records that have 90% of the same key names, and suggest them as similar enough to merit treating as instances of the same type. (Note that this is optional, and can be tuned manually.)\n3. Last step is to derive unique-ish type names - we currently do it by concatenating the name of the container and name of the key. (Please open PR, if you want something fancy about that - initial version used just key name, when it was unique.)\n4. Finally it generates [Haskell](https://www.haskell.org/) or [Elm](http://elm-lang.org/) code for the type.\n\nCombination of robust [*union type system*](https://github.com/mgajda/json-autotype/blob/master/Data/Aeson/AutoType/Type.hs), and heuristic makes this system extremely reliable.\nMain test is QuickCheck-based generation of random JSON documents, and checking that they are all correctly parsed by resulting parser.\n\nMore details are described in [Haskell.SG meetup presentation](https://engineers.sg/video/json-autotype-1-0-haskell-sg--429).\n\nOther approaches:\n=================\n\n* There is a [TypeScript type provider](https://jvilk.com/MakeTypes/), and [PLDI 2016 paper](https://dl.acm.org/citation.cfm?id=2908115) on solving this problem using \u003cem\u003epreferred type shapes\u003c/em\u003e instead of union types.\nOne can think about it as a alternative theory that gives very similar results, with more complicated exposition. It also does not tackle the problem of tagged records. It also does not attempt to \u003cem\u003eguess\u003c/em\u003e unification candidates in order to reduce type complexity.\n* There *was* a [json-sampler](https://maxs.io/generating-types-from-json-samples/) that allows to make simpler data structure from JSON examples, but doesn't seem to perform unification, nor is it suitable for big APIs.\n\n* [PADS project](https://www.cs.princeton.edu/~dpw/papers/padsml06.pdf) is another attempt to automatically infer types to treat \u003cem\u003earbitrary\u003c/em\u003e data formats (not just JSON). It mixes type declarations, with parsing/printing information in order to have a consistent view of both. It does not handle automatic type inference though.\n* [JSON Schema generator](https://www.newtonsoft.com/jsonschema/help/html/GenerateSchema.htm) uses .NET types to generate JSON Schema instead (in opposite direction.) Similar schema generation is [used here](https://sixgun.wordpress.com/2012/02/09/using-json-net-to-generate-jsonschema/)\n* Microsoft Developer Network advocates use of [Data Contracts](https://docs.microsoft.com/en-us/dotnet/framework/wcf/feature-details/using-data-contracts) instead to constrain possible input data.\n* [QuickType](https://github.com/quicktype/quicktype) uses [Markov chains heuristic](https://blog.quicktype.io/markov/) instead of theory\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmigamake%2Fjson-autotype","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmigamake%2Fjson-autotype","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmigamake%2Fjson-autotype/lists"}