{"id":15491774,"url":"https://github.com/tristanpenman/json-preprocessor","last_synced_at":"2025-10-12T04:32:06.591Z","repository":{"id":56325964,"uuid":"312461470","full_name":"tristanpenman/json-preprocessor","owner":"tristanpenman","description":"JSON Preprocessor library / CLI for constructing JSON documents from templates, written in Python","archived":true,"fork":false,"pushed_at":"2022-02-22T01:45:56.000Z","size":39,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-10-19T10:23:52.056Z","etag":null,"topics":["experimental","json"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tristanpenman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-11-13T03:19:12.000Z","updated_at":"2023-12-19T23:55:47.000Z","dependencies_parsed_at":"2022-08-15T16:40:42.968Z","dependency_job_id":null,"html_url":"https://github.com/tristanpenman/json-preprocessor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanpenman%2Fjson-preprocessor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanpenman%2Fjson-preprocessor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanpenman%2Fjson-preprocessor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tristanpenman%2Fjson-preprocessor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tristanpenman","download_url":"https://codeload.github.com/tristanpenman/json-preprocessor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236166118,"owners_count":19105807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["experimental","json"],"created_at":"2024-10-02T07:56:19.829Z","updated_at":"2025-10-12T04:32:01.332Z","avatar_url":"https://github.com/tristanpenman.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# JSON Preprocessor\n\nThis project provides a JSON Preprocessor library and command line utility that can be used to resolve JSON References and other preprocessor directives. It was originally motivated by the need to construct CloudFormation templates as part of a CI pipeline.\n\nThe following directives are supported:\n\n* ```$exec```\n* ```$join```\n* ```$merge```\n* ```$ref```\n\n*Note that JSON Reference support is not yet complete.*\n\nThis project has been tested against Python 2.7 and 3.5.\n\n## Directives\n\n### $exec\n\nThe `$exec` directive allows the output of an external command to be included as a string value in a JSON document.\n\nThe command to be executed is provided as an array. The first element should be the path to an executable, with the remaining elements containing program arguments.\n\nFor example, take the following JSON snippet:\n\n    {\n        \"timestamp\": {\n            \"$exec\": [ \"/bin/date\" ]\n        }\n    }\n\nResolving this directive would produce something like this:\n\n    {\n        \"timestamp\": \"Thu 25 Sep 2014 15:30:40 AEST\"\n    }\n\nIt is important to note that an `$exec` directive is *not* executed in a shell, meaning that pipes, redirection and other shell built-ins are not available.\n\n### $join\n\nThe input for a `$join` directive must be an array containing two elements. The first element is an array of strings (or directives that can be resolved to strings). The second element is a string that will be placed between adjacent elements in the output string.\n\nHere is an example in which a `$join` directive is used to concatenate three string values:\n\n    {\n        \"$join\": [ [\"A\", \"B\", \"C\"], \" \" ]\n    }\n\nWhen resolved, this example would produce the following output:\n\n    \"A B C\"\n\nThis can be useful when constructing strings using the output of other directives. Here is example that joins a static string with the output of an `$exec` directive:\n\n    {\n        \"$join\": [ [ \"Current time:\", { \"$exec\": [ \"/bin/date\" ] } ], \" \" ]\n    }\n\nThis would be produce something like this:\n\n    \"Current time: Thu 25 Sep 2014 15:30:40 AEST\"\n\nAlternatively, an array delimiter can be used, which will change `$join` to perform array concatenation:\n\n    {\n        \"$join\": [\n            [\n                { \"$ref\": \"file://abc.json\" },\n                [ \"1\", \"2\", \"3\" ],\n                { \"$ref\": \"file://xyz.json\" }\n            ],\n            [ \"*\", \"*\" ]\n        ]\n    }\n\nAssuming `abc.json` and `xyz.json` contain the arrays `[\"a\", \"b\", \"c\"]` and `[\"x\", \"y\", \"z\"]` respectively, the output would look like this:\n\n    [ \"a\", \"b\", \"c\", \"*\", \"*\", \"1\", \"2\", \"3\", \"*\", \"*\", \"x\", \"y\", \"z\" ]\n\n### $merge\n\nThe input for a `$merge` directive must be an array of objects. These objects will be combined, with the attributes of later objects taking precedence over those provided by earlier objects.\n\nHere is an example of a `$merge` directive that combines two objects:\n\n    {\n        \"$merge\": [\n            {\n                \"my_attribute\": \"original_value\",\n                \"another_attribute\": \"another_value\"\n            },\n            {\n                \"my_attribute\": \"replacement_value\"\n            }\n        ]\n    }\n\nWhen resolved, this example `$merge` directive would produce the following output:\n\n    {\n        \"my_attribute\": \"replacement_value\",\n        \"another_attribute\": \"another_value\"\n    }\n\nNote that the value of `my_attribute` defined by the first object has been replaced by the value defined by the second object.\n\n### $ref\n\nA `$ref` directive allows JSON References, and other remote resource references, to be resolved.\n\nThe JSON Preprocessor library has built-in support for the following resource types:\n\n* `http://`\n* `https://`\n* `file://` (for absolute file references)\n* `rel://` (for relative file references)\n\nJSON References can be used to embed the all or part of an external JSON document in the preprocessor output. The library also allows custom resource types to be supported.\n\n## JSON Preprocessor Utility\n\nThis project includes a JSON Preprocessor command line utility (see [json_preprocessor/cli.py](./json_preprocessor/cli.py)) that serves as an example of how to use the JSON Preprocessor library. It extends the `$ref` directive to retrieve attributes associated with CloudFormation resources, which can be useful when constructing CloudFormation templates as part of an automated build process.\n\n### CFN References\n\nThis example registers a custom `$ref` URI type that adds support for CloudFormation resources via the [boto](https://github.com/boto/boto) library.\n\nCloudFormation resources are identified using a specific URI format:\n\n    cfn://\u003cstack-name\u003e[@region]/\u003clogical-name\u003e[/[attribute]]\n\n`[]` denotes optional components, whereas `\u003c\u003e` denotes mandatory components.\n\nIf any mandatory components are missing, an exception will be raised.\n\nIf `[region]` is omitted, then the region will be determined using the current user's AWS credentials. If `[region]` is present, but not recognised by boto, an exception may be raised.\n\n`[attribute]` may be any attribute that can be returned by the retrieval function. If `[attribute]` is omitted, the 'PhysicalResourceID' attribute will be returned.\n\n### Usage\n\n    Usage: json-preprocessor [OPTIONS] \u003cpath-to-document\u003e COMMAND [ARGS]...\n\n      Resolve a CloudFormation template containing JSON pre-processor\n      directives.\n\n    Options:\n      --minify                 Compact the JSON output by removing whitespace.\n      --output-file \u003cpath\u003e     Optional path to which JSON output will be written.\n                               By default output will be written to STDOUT.\n      --parameter \u003ckey=value\u003e  A key-value pair to be passed to the template; this\n                               option may be used more than once to pass in\n                               multiple key-value pairs.\n      --help                   Show this message and exit.\n\n## Installation\n\nBoth the library and an example command line utility can be installed using pip:\n\n    pip install -r requirements\n    pip install .\n\nYou can check that the installation was successful by running the command line\nutility with no arguments:\n\n    json-preprocessor\n\nIf the installation was successful, usage instructions for `json-preprocessor` will be displayed.\n\n## Development\n\nThe json_preprocessor module also includes a `__main__.py` file, so an alternative is to run the example locally:\n\n    pip install -r requirements\n    python -m json_preprocessor\n\nAnd to run the tests:\n\n    python -m unittest\n\n## License\n\nThis code is licensed under the 3-clause BSD License.\n\nSee the LICENSE file for more information.\n\n## Acknowledgements\n\nThanks to [elruwen](https://github.com/elruwen) for early code reviews, and [cvlvxi](https://github.com/cvlvxi) for help modernising the code.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftristanpenman%2Fjson-preprocessor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftristanpenman%2Fjson-preprocessor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftristanpenman%2Fjson-preprocessor/lists"}