{"id":13581426,"url":"https://github.com/miracl/conflate","last_synced_at":"2025-04-13T22:50:28.240Z","repository":{"id":38291102,"uuid":"116945174","full_name":"miracl/conflate","owner":"miracl","description":"Library providing routines to merge and validate JSON, YAML and/or TOML files","archived":false,"fork":false,"pushed_at":"2024-07-22T14:53:34.000Z","size":5119,"stargazers_count":46,"open_issues_count":10,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-27T13:05:30.192Z","etag":null,"topics":["config","configuration","configuration-management","golang","golang-library","json","json-schema","merge","structs","toml","yaml"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/miracl.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-01-10T10:51:02.000Z","updated_at":"2024-10-22T02:05:41.000Z","dependencies_parsed_at":"2024-01-15T17:39:26.864Z","dependency_job_id":"0b234968-57e0-497f-a584-9df298991ec8","html_url":"https://github.com/miracl/conflate","commit_stats":{"total_commits":65,"total_committers":4,"mean_commits":16.25,"dds":0.3076923076923077,"last_synced_commit":"313a23bf402f17384e9b4cd4c551d95d5c7586ca"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracl%2Fconflate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracl%2Fconflate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracl%2Fconflate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/miracl%2Fconflate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/miracl","download_url":"https://codeload.github.com/miracl/conflate/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248794560,"owners_count":21162614,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","configuration","configuration-management","golang","golang-library","json","json-schema","merge","structs","toml","yaml"],"created_at":"2024-08-01T15:02:01.513Z","updated_at":"2025-04-13T22:50:28.204Z","avatar_url":"https://github.com/miracl.png","language":"Go","readme":"\u003cp align=\"center\"\u003e\u003cimg src=\"gophers.png\" alt=\"gophers\" style=\"width: 50%; height: 50%\"\u003e\u003c/p\u003e\n\n# CONFLATE\n\n_Library providing routines to merge and validate JSON, YAML, TOML files and/or structs ([godoc](https://godoc.org/github.com/miracl/conflate))_\n\n_Typical use case: Make your application configuration files **multi-format**, **modular**, **templated**, **sparse**, **location-independent** and **validated**_\n\n[![Build Status](https://secure.travis-ci.org/miracl/conflate.png?branch=master)](https://travis-ci.org/miracl/conflate?branch=master)\n[![Coverage Status](https://coveralls.io/repos/miracl/conflate/badge.svg?branch=master\u0026service=github)](https://coveralls.io/github/miracl/conflate?branch=master)\n[![Go Report Card](https://goreportcard.com/badge/github.com/miracl/conflate)](https://goreportcard.com/report/github.com/miracl/conflate)\n\n## Description\n\nConflate is a library and cli-tool, that provides the following features :\n\n* merge data from multiple formats (JSON/YAML/TOML/go structs) and multiple locations (filesystem paths and urls)\n* validate the merged data against a JSON schema\n* apply any default values defined in a JSON schema to the merged data\n* expand environment variables inside the data\n* marshal merged data to multiple formats (JSON/YAML/TOML/go structs)\n\nIt supports draft-04, draft-06 and draft-07 of JSON Schema. If the key $schema is missing, or the draft version is not explicitly set, a hybrid mode is used which merges together functionality of all drafts into one mode.\nImprovements, ideas and bug fixes are welcomed.\n\n## Getting started\n\nRun the following command, which will build and install the latest binary in $GOPATH/bin\n\n```\ngo install github.com/miracl/conflate/...@latest\n```\nAlternatively, you can install one of the pre-built release binaries from https://github.com/miracl/conflate/releases\n\n## Usage of Library\n\nPlease refer to the [godoc](https://godoc.org/github.com/miracl/conflate) and the [example code](./example/main.go)\n\n## Usage of CLI Tool\n\nHelp can be obtained in the usual way :\n\n```bash\n$conflate --help\nUsage of conflate:\n  -data value\n    \tThe path/url of JSON/YAML/TOML data, or 'stdin' to read from standard input\n  -defaults\n    \tApply defaults from schema to data\n  -expand\n    \tExpand environment variables in files\n  -format string\n    \tOutput format of the data JSON/YAML/TOML\n  -includes string\n    \tName of includes array. Blank string suppresses expansion of includes arrays (default \"includes\")\n  -noincludes\n    \tSwitches off conflation of includes. Overrides any --includes setting.\n  -schema string\n    \tThe path/url of a JSON v4 schema file\n  -validate\n    \tValidate the data against the schema\n  -version\n    \tDisplay the version number\n```\n\nTo conflate the following file ... :\n\n```bash\n$cat ./testdata/valid_parent.json\n{\n  \"includes\": [\n    \"valid_child.json\",\n    \"valid_sibling.json\"\n  ],\n  \"parent_only\" : \"parent\",\n  \"parent_child\" : \"parent\",\n  \"parent_sibling\" : \"parent\",\n  \"all\": \"parent\"\n}\n```\n\n...run the following command, which will merge [valid_parent.json](https://raw.githubusercontent.com/miracl/conflate/master/testdata/valid_parent.json),\n[valid_child.json](https://raw.githubusercontent.com/miracl/conflate/master/testdata/valid_child.json), [valid_sibling.json](https://raw.githubusercontent.com/miracl/conflate/master/testdata/valid_sibling.json)  :\n\n```bash\n$conflate -data ./testdata/valid_parent.json -format JSON\n{\n  \"all\": \"parent\",\n  \"child_only\": \"child\",\n  \"parent_child\": \"parent\",\n  \"parent_only\": \"parent\",\n  \"parent_sibling\": \"parent\",\n  \"sibling_child\": \"sibling\",\n  \"sibling_only\": \"sibling\"\n}\n```\nNote how the `includes` are loaded remotely as relative paths.\n\nAlso, note values in a file override values in any included files, and that an included file overrides values in any included file above it in the `includes` list.\n\nIf you instead host a file somewhere else, then just use a URL :\n\n```bash\n$conflate -data https://raw.githubusercontent.com/miracl/conflate/master/testdata/valid_parent.json -format JSON\n{\n  \"all\": \"parent\",\n  \"child_only\": \"child\",\n  \"parent_child\": \"parent\",\n  \"parent_only\": \"parent\",\n  \"parent_sibling\": \"parent\",\n  \"sibling_child\": \"sibling\",\n  \"sibling_only\": \"sibling\"\n}\n\n```\n\nThe `includes` here are also loaded as relative urls and follow exactly the same merging rules.\n\nTo output in a different format use the `-format` option, e.g. TOML :\n\n```bash\n$conflate -data ./testdata/valid_parent.json -format TOML\nall = \"parent\"\nchild_only = \"child\"\nparent_child = \"parent\"\nparent_only = \"parent\"\nparent_sibling = \"parent\"\nsibling_child = \"sibling\"\nsibling_only = \"sibling\"\n```\n\nTo additionally use defaults from a JSON [schema](https://raw.githubusercontent.com/miracl/conflate/master/testdata/test.schema.json) and validate the conflated data against the schema, use `-defaults` and `-validate` respectively :\n\n```bash\n$cat ./testdata/blank.yaml\n\n$conflate -data ./testdata/blank.yaml -schema ./testdata/test.schema.json -validate -format YAML\nSchema validation failed : The document is not valid against the schema : Invalid type. Expected: object, given: null (#)\n\n$conflate -data ./testdata/blank.yaml -schema ./testdata/test.schema.json -defaults -validate -format YAML\nall: parent\nchild_only: child\nparent_child: parent\nparent_only: parent\nparent_sibling: parent\nsibling_child: sibling\nsibling_only: sibling\n```\n\nNote any defaults are applied before validation is performed, as you would expect.\n\nIf you don't want to intrusively embed an `\"includes\"` array inside your JSON, you can instead provide multiple data files which are processed from left-to-right :\n\n```bash\n$conflate -data ./testdata/valid_child.json -data ./testdata/valid_sibling.json -format JSON\n{\n  \"all\": \"sibling\",\n  \"child_only\": \"child\",\n  \"parent_child\": \"child\",\n  \"parent_sibling\": \"sibling\",\n  \"sibling_child\": \"sibling\",\n  \"sibling_only\": \"sibling\"\n}\n```\n\nOr alternatively, you can create a top-level JSON file containing only the `includes` array. For fun, lets choose to use YAML for the top-level file, and output TOML :\n\n```bash\n$cat toplevel.yaml\nincludes:\n  - testdata/valid_child.json\n  - testdata/valid_sibling.json\n\n$conflate -data toplevel.yaml -format TOML\nall = \"sibling\"\nchild_only = \"child\"\nparent_child = \"child\"\nparent_sibling = \"sibling\"\nsibling_child = \"sibling\"\nsibling_only = \"sibling\"\n```\n\nIf you want to read a file from stdin you can do the following. Here we pipe in some TOML to override a value to demonstrate :\n\n```bash\n$echo 'all=\"MY OVERRIDDEN VALUE\"' |  conflate -data ./testdata/valid_parent.json -data stdin  -format JSON\n{\n  \"all\": \"MY OVERRIDDEN VALUE\",\n  \"child_only\": \"child\",\n  \"parent_child\": \"parent\",\n  \"parent_only\": \"parent\",\n  \"parent_sibling\": \"parent\",\n  \"sibling_child\": \"sibling\",\n  \"sibling_only\": \"sibling\"\n}\n```\n\nNote that in all cases `-data` sources are processed from left-to-right, with values in right files overriding values in left files, so the following doesn't work :\n\n```bash\n$echo 'all=\"MY OVERRIDDEN VALUE\"' |  conflate -data stdin -data ./testdata/valid_parent.json  -format JSON\n{\n  \"all\": \"parent\",\n  \"child_only\": \"child\",\n  \"parent_child\": \"parent\",\n  \"parent_only\": \"parent\",\n  \"parent_sibling\": \"parent\",\n  \"sibling_child\": \"sibling\",\n  \"sibling_only\": \"sibling\"\n}\n```\n\nYou can optionally expand environment variables in the files like this :\n\n```bash\n$export echo MYVALUE=\"some value\"\n$export echo MYJSONMAP='{ \"item1\" : \"value1\" }'\n$echo '{ \"my_value\": \"$MYVALUE\", \"my_map\": $MYJSONMAP }' | conflate -data stdin -expand -format JSON\n{\n  \"my_map\": {\n    \"item1\": \"value1\"\n  },\n  \"my_value\": \"some value\"\n}\n```\n\n# Acknowledgements\n\nImages derived from originals by Renee French https://golang.org/doc/gopher/\n","funding_links":[],"categories":["Go","Configuration"],"sub_categories":["Advanced Console UIs"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracl%2Fconflate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmiracl%2Fconflate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmiracl%2Fconflate/lists"}