{"id":13489509,"url":"https://github.com/TomConlin/json_to_paths","last_synced_at":"2025-03-28T05:31:01.135Z","repository":{"id":61126793,"uuid":"194812798","full_name":"TomConlin/json_to_paths","owner":"TomConlin","description":"Distill a JSON document into a collection of paths both for 'jq' and 'xpath' ","archived":false,"fork":false,"pushed_at":"2021-02-20T01:04:02.000Z","size":151,"stargazers_count":106,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-10-31T03:33:26.966Z","etag":null,"topics":["jq","json","xpaths"],"latest_commit_sha":null,"homepage":"","language":"JSONiq","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TomConlin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-02T07:36:32.000Z","updated_at":"2024-09-25T20:25:33.000Z","dependencies_parsed_at":"2022-10-11T11:17:05.103Z","dependency_job_id":null,"html_url":"https://github.com/TomConlin/json_to_paths","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TomConlin%2Fjson_to_paths","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TomConlin%2Fjson_to_paths/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TomConlin%2Fjson_to_paths/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TomConlin%2Fjson_to_paths/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TomConlin","download_url":"https://codeload.github.com/TomConlin/json_to_paths/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245978200,"owners_count":20703675,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jq","json","xpaths"],"created_at":"2024-07-31T19:00:29.429Z","updated_at":"2025-03-28T05:31:00.477Z","avatar_url":"https://github.com/TomConlin.png","language":"JSONiq","funding_links":[],"categories":["JSONiq"],"sub_categories":[],"readme":"\n### JSON to `jq` paths  or XML xpath\n\n#### Purpose\nTo get insight into the structure of arbitrary JSON files from the command line.\nProvides building blocks for further processing both with 'jq' as well as other tools\nintended to be used with XML xpath.\n\n\n\n### Requirements\n\n - `jq` installed see https://github.com/stedolan/jq/\n(Most likely installs with your package manager.)\n\n - the one liner jq scripts this ropo is for\n\n#### Usage\n\nSnagging a random (to me) test JSON file from a neat looking\n[awk json parser project](https://github.com/step-/JSON.awk)\n\n```\ncurl -sL https://github.com/step-/JSON.awk/raw/master/test-cases/20170131-issue-007-test.json \u003e testdata.json\n```\n### Result from generating `jq` paths\n```\n ./json2jqpath.jq testdata.json \u003e testdata.jqpath\n```\nNote: making up the format suffix `.jqpath`\n\n```\ncat testdata.jqpath\n.\n.response\n.response|.count\n.response|.data\n.response|.data|.[]\n.response|.data|.[]|.person\n.response|.data|.[]|.person|.address\n.response|.data|.[]|.person|.age\n.response|.data|.[]|.person|.balance\n.response|.data|.[]|.person|.company\n.response|.data|.[]|.person|.coords\n.response|.data|.[]|.person|.coords|.lat\n.response|.data|.[]|.person|.coords|.long\n.response|.data|.[]|.person|.email\n.response|.data|.[]|.person|.gender\n.response|.data|.[]|.person|.guid\n.response|.data|.[]|.person|.id\n.response|.data|.[]|.person|.isActive\n.response|.data|.[]|.person|.name\n.response|.data|.[]|.person|.phone\n.response|.data|.[]|.person|.picture\n.response|.data|.[]|.person|.registered\n.response|.data|.[]|.person|.tags\n.response|.data|.[]|.person|.tags|.[]\n.response|.pagecount\n\n```\nThis is a succinct representation of every simple path available through the json structure.\n\n\nExample:\nTo see what the distribution of most common `tags` in that file.\n\nNoting that the path agrument to the next `jq`  call  \n   **\".response|.data|.[]|.person|.tags|.[]\"**  \nis cut-n-paste from the next to last line of output from\n`json2jqpath.jq`  above.\n\n\n```\njq \".response|.data|.[]|.person|.tags|.[]\" testdata.json |\n    sort | uniq -c | sort -nr | head\n     75 \"est\"\n     64 \"labore\"\n     63 \"consectetur\"\n     58 \"occaecat\"\n     57 \"fugiat\"\n     57 \"excepteur\"\n     56 \"proident\"\n     56 \"Lorem\"\n     56 \"laborum\"\n     56 \"incididunt\"\n```\n\nLooks like they used [Lorem Ipsum](https://en.wikipedia.org/wiki/Lorem_ipsum) as filler.\n\nTry for something more specific such as person's name and location.\n```\njq \".response|.data|.[]|.person|.name,.coords\" testdata.json | head\n\"Anastasia Goodwin\"\n{\n  \"lat\": 71.050828,\n  \"long\": 113.565478\n}\n\"Peters Watson\"\n{\n  \"lat\": 0.203464,\n  \"long\": -71.896296\n}\n\n```\n\n#### Result for genreating XML xpaths\n```\n./json2xpath.jq testdata.json| sort -u \u003e testdata.xpath\ncat testdata.xpath\n./response\n./response/count\n./response/data\n./response/data/person\n./response/data/person/address\n./response/data/person/age\n./response/data/person/balance\n./response/data/person/company\n./response/data/person/coords\n./response/data/person/coords/lat\n./response/data/person/coords/long\n./response/data/person/email\n./response/data/person/gender\n./response/dfromata/person/guid\n./response/data/person/id\n./response/data/person/isActive\n./response/data/person/name\n./response/data/person/phone\n./response/data/person/picture\n./response/data/person/registered\n./response/data/person/tags\n./response/pagecount\n```\n\nwhich is a good starting point for tools such as [xpath2dot.awk](https://github.com/TomConlin/xpath2dot)\n\n\n```\n./json2xpath.jq testdata.json |\n    sort -u |\n    xpath2dot.awk -v ORIENT=\"UD\" |\n    dot -T svg \u003e testdata.svg\n\n```\n\n![testdata.svg](https://github.com/TomConlin/json2xpath/blob/master/testdata.svg)\n\n\n--------\n\nA note on the use of `sort -u` above\n\nThe script `json2xpath.jq` has the ability to sort and remove duplicates but they\nare [commented out](https://github.com/TomConlin/json2xpath/blob/master/json2xpath.jq#L14).\n\nThey are disabled by default to allow more use cases\nfor example:\n\n - We may be extracting data in some other program \u0026 want to know the native order\n - We may be interested in counts of things of various types\n\nIf you know these are not your use case you can uncomment them in the script\nand eliminate piping the output through  `sort -u`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTomConlin%2Fjson_to_paths","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTomConlin%2Fjson_to_paths","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTomConlin%2Fjson_to_paths/lists"}