{"id":22707523,"url":"https://github.com/xvik/yaml-updater","last_synced_at":"2025-04-13T12:31:48.304Z","repository":{"id":38109974,"uuid":"400149775","full_name":"xvik/yaml-updater","owner":"xvik","description":"Yaml configuration merge tool","archived":false,"fork":false,"pushed_at":"2025-03-13T05:20:04.000Z","size":736,"stargazers_count":11,"open_issues_count":3,"forks_count":5,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-27T03:35:13.029Z","etag":null,"topics":["config","dropwizard","migration-tool","yaml"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xvik.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-26T11:49:07.000Z","updated_at":"2025-03-13T05:20:01.000Z","dependencies_parsed_at":"2023-02-03T23:15:18.021Z","dependency_job_id":"38ee3f28-92c2-4045-a5fa-43d12ff5c4dd","html_url":"https://github.com/xvik/yaml-updater","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvik%2Fyaml-updater","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvik%2Fyaml-updater/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvik%2Fyaml-updater/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvik%2Fyaml-updater/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xvik","download_url":"https://codeload.github.com/xvik/yaml-updater/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248714485,"owners_count":21149905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","dropwizard","migration-tool","yaml"],"created_at":"2024-12-10T10:13:20.539Z","updated_at":"2025-04-13T12:31:48.272Z","avatar_url":"https://github.com/xvik.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Yaml config updater\n[![License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](http://www.opensource.org/licenses/MIT)\n[![CI](https://github.com/xvik/yaml-updater/actions/workflows/CI.yml/badge.svg)](https://github.com/xvik/yaml-updater/actions/workflows/CI.yml)\n[![Appveyor build status](https://ci.appveyor.com/api/projects/status/github/xvik/yaml-updater?svg=true)](https://ci.appveyor.com/project/xvik/yaml-updater)\n[![codecov](https://codecov.io/gh/xvik/yaml-updater/branch/master/graph/badge.svg)](https://codecov.io/gh/xvik/yaml-updater)\n\nSupport: [Discussions](https://github.com/xvik/yaml-updater/discussions) | [Gitter chat](https://gitter.im/xvik/yaml-updater)\n\n### About\n\nMerges yaml configuration files, preserving comments and whitespaces. Assumed to be used for microservice configuration\nupdates.\n\nBy default, targets the most common use-case: add all new properties, without removing or changing existing values.\nLists not merged because list is a value and all current values must remain.\n\n[Introduction article](https://blog.vyarus.ru/yaml-confg-migration-tool)\n\nComments preserved using custom (simple) yaml parser. [Snakeyaml](https://bitbucket.org/asomov/snakeyaml/wiki/Documentation) \nparser used for source files validation, comments parser self-validation (compares parse trees)\nand result validation.\n\nDue to complete validation, merged file correctness is guaranteed.\n\nSupports:\n\n* Multiline values (all [syntax variations](https://yaml-multiline.info))\n* [Reformatting](#merge-rules) (changed paddings in new config in both directions)\n* Quoted property names with escaped characters (according to [yaml spec](http://yaml.org/spec/1.2-old/spec.html#id2776092))\n* Properties reordering according to new config\n* [Variables replacement](#variables) in new config before merge (environment-specific config adoption)\n* [Current values remove](yaml-config-updater#delete-props) (e.g. deprecated values or for value replacement)\n* [Object list items update](#lists-matching-logic) (lists not merged, but new properties could be added to list items)\n* [Backup](yaml-config-updater#backup) current configuration (only if content changes)\n* [Migration testing](yaml-config-updater#testing) (dry run in cli and testing api)\n\nIMPORTANT: this is not a general-purpose yaml merge tool because yaml features like multiple documents in\none file and object references are not supported (only common subset of features, used in configs)\n\n### Example\n\nOriginal config:\n\n```yaml\n# top comment\n\n# something\nprop:\n  one: 1\n\n  two: 2\n\nlists:\n\n  # sub comment\n  list:\n    - one\n    - two\n\n  obj:\n    - one: 1\n      two: 2\n\nlarge: multi-line\n  value\n\n# trailing comment\n```\n\nUpdate file:\n\n```yaml\n# changed comment\n\n# something else\nprop:\n  one: 3\n  two: 3\n  three: 3                              # new property\n\nlists:\n\n  # changed sub comment\n  list:                                 # padding changed\n      - two                             # order changed (ignored)\n      - one\n      - three                           # new value ignored\n\n  obj:\n    - two: 2\n      one: 1\n      three: 3                        # new value\n\nlarge: multi-line\n  value\n\n# changed trailing comment\n```\n\nMerge result:\n\n```yaml\n# changed comment\n\n# something else\nprop:\n  one: 1\n\n  two: 2\n  three: 3                              # new property\n\nlists:\n\n  # changed sub comment\n  list:\n      - one\n      - two\n\n  obj:\n    - two: 2\n      one: 1\n      three: 3                        # new value\n\nlarge: multi-line\n  value\n\n# changed trailing comment\n```\n\nMerge report (shown by cli modules):\n\n```\nConfiguration: /var/somewhere/config.yml (185 bytes, 23 lines)\nUpdated from source of 497 bytes, 25 lines\nResulted in 351 bytes, 25 lines\n\n\tAdded from new file:\n\t\tprop/three                               7  | three: 3                              # new property\n\t\tlists/obj[0]/three                       20 | three: 3                        # new value\n```\n\n### Setup\n\n[![Maven Central](https://img.shields.io/maven-central/v/ru.vyarus/yaml-config-updater.svg?style=flat)](https://maven-badges.herokuapp.com/maven-central/ru.vyarus/yaml-config-updater)\n\nCould be used as:\n\n* [Library](yaml-config-updater) through API (embedding tool or perform testing migration)\n* [Command line util](yaml-config-updater-cli) (fat jar, native binary or docker image)\n* [Dropwizard plugin](dropwizard-config-updater) (same arguments as in cli module)\n\nRead exact module's readme for setup instructions.\n\n### Merge rules\n\n* All values from current configuration preserved \n* All new properties added\n* Order of properties from update file used (current config properties could be re-ordered)\n* For existing values, top comments updated from new config (if property exists in new config).\n  - If new property does not contain any comment - old comment is preserved\n  - In-line comments (after value) not updated (and so may be used for persistent marks)\n  - Trailing comment at the end of file preserved as-is (not updated)\n* Paddings applied from update file (original config could be re-formatted)\n  - Padding change works in both directions (reduce or enlarge)\n* Possible whitespace between property name and colon is removed (`name : value` becomes `name: value`)\n* Property writing style taken from target file: e.g. if property was quoted and not quoted in new file then it would be not quoted in merged file\n* Lists not merged\n  - Object list items could be updated with new properties (if matched item found)\n  - Dash style for object item could change in both directions: empty dash (property on new line) or property on the same line with dash\n* Flow style lists and objects considered as single value and not merged: `[1, 2, 3]` or `{ one:1, two: 2 }`\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003eCase\u003c/th\u003e\n\u003cth\u003eCurrent config\u003c/th\u003e\n\u003cth\u003eUpdate file\u003c/th\u003e\n\u003cth\u003eMerge result\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\nNew properties added\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\ntwo: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: 3\ntwo: 3\nthree: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: 1\ntwo: 2\nthree: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nOrder changed\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\ntwo: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nthree: 3\ntwo: 3\none: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nthree: 3\ntwo: 2\none: 1\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nPadding changed\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: \n  two: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: \n    two: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: \n    two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nComment updated\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: \n  # Old comment\n  two: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: \n    # New comment\n    two: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: \n    # New comment\n    two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nStyle changed\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\n\"one\": 1\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: 1\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\none: 1\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nList not merged, but padding updated\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\nlist: \n  - one\n  - two\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n    - one\n    - three\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n    - one\n    - two\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nObject list item updated\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\nlist: \n  - one: 1\n    two: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n  - one: 1\n    two: 2\n    three: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n  - one: 1\n    two: 2\n    three: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nList declaration style could change\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\nlist: \n  - one: 1\n    two: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n  - \n    one: 1\n    two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\nlist: \n  - \n    one: 1\n    two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### Lists matching logic\n\nObject list items updated because in some cases lists could be quite complex and contain \nupdatable configuration blocks.\n\nItems match is searched by \"the most similarity\": selects item containing\nmore properties with the same value.\n\n* Item containing property with different value would never match\n* If item contains sub-lists then at least one list item in current list must match\n    - Scalar list values ignored (only object lists counted)\n* Only exact match counts: if multiple items match with the same amount of matched properties\n  then no item would be updated (avoid guessing, work only on exact matches)\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003cth\u003eCase\u003c/th\u003e\n\u003cth\u003eList item\u003c/th\u003e\n\u003cth\u003eCandidates\u003c/th\u003e\n\u003cth\u003eMatch\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\nMatch by value\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\ntwo: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\n- one: 1\n  two: 3\n- one: 1\n  two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n Item 2\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nMatch by \"weight\"\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\ntwo: 2\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\n- one: 1\n  three: 3\n- one: 1\n  two: 2\n  three: 3\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n Item 2 (technically both matches, but first item with one and second item with 2 props)\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nMultiple matches\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\n- one: 1\n  two: 3\n- one: 1\n  two: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n No\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nScalar list ignored\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\nsub:\n    - a\n    - b\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\n- one: 1\n  sub:\n    - c\n    - d\n- one: 1\n  sub:\n    - e\n    - f\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n No (both matches, scalar list values ignored)\n\u003c/td\u003e\n\u003c/tr\u003e\n\n\u003ctr\u003e\n\u003ctd\u003e\nSub list (one sub list element matched)\n\u003c/td\u003e\n\u003ctd\u003e\n   \u003cpre lang=\"yaml\"\u003e\none: 1\nsub:\n    - a: 1\n   \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n  \u003cpre lang=\"yaml\"\u003e\n- one: 1\n  sub:\n    - a: 2\n    - a: 1\n- one: 1\n  sub:\n    - b: 1\n    - b: 2\n  \u003c/pre\u003e\n\u003c/td\u003e\n\u003ctd\u003e\n Item 1 (sub list has (at least) one match) \n\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n### Variables\n\nUpdate config could contain variables with syntax: `#{name}`.\nSuch syntax used instead of more common `${..}` because:\n\n- Config may rely on environment variables and so will collide with replacing variables\n- Such syntax recognized as comment and so not processed value would be empty value after parsing.\n\n```yaml\nobj:\n  some: #{name}\n```\n\nIMPORTANT: only known placeholders would be replaced (unknown variables will remain unprocessed)\n\nFeature supposed to be used to \"adopt\" config for current environment before processing.\nCli modules would automatically use environment variables.\n\n### General workflow\n\n* Read update config\n  - replace variables\n  - parse with snakeyaml (validation)\n  - parse with comments parser\n  - compare trees (self-validation)\n* Read current config\n  - if not exists, simply copy update file\n  - parse with snakeyaml and comments parsers and compare trees\n* Perform merge\n* Write to tmp file\n* Read merged result with snakeyaml (validate file correctness)\n* Check all new properties added and current values preserved (validation)\n* Backup current config (in the same folder)\n* Replace config\n\nTwo last steps performed only if file content changes, otherwise current config remain untouched (to avoid producing redundant backups).\n\n### Comments parser specifics\n \nParser assume everything above property as property comment (even empty lines):\n\n```yaml\n# Large comment\n\n# With empty lines\nprop: 1\n```\n\nExactly because of this comment is always replaced from new file.\n\nFor example, assume reordering case:\n\nOriginal config:\n\n```yaml\n# Large header\n\n# property comment\nprop1: 1\nprop2: 2\n```\n\nNew config:\n\n```yaml\n# Large header\n\nprop2: 2\n# property comment\nprop1: 1\n```\n\nWithout using comments from new file, entire header would be moved below the first property,\nbut with comments update header would remain untouched.\n\nBut still, some edge cases possible (with removed properties).\n\n#### Values\n\nParser stores property values as everything after colon (including possible in-line comments).\nThe same for multi-line values. \n\nThis way, current value could be restored *exactly* the same as it was in the original file\n(to avoid re-formatting).\n\nAnd because of this in-line comments are not recognized and so could \"survive\" update.\n\n---\n[![java lib generator](http://img.shields.io/badge/Powered%20by-%20Java%20lib%20generator-green.svg?style=flat-square)](https://github.com/xvik/generator-lib-java)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxvik%2Fyaml-updater","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxvik%2Fyaml-updater","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxvik%2Fyaml-updater/lists"}