{"id":52037200,"url":"https://github.com/adobe/himl","last_synced_at":"2026-08-01T21:01:57.262Z","repository":{"id":37546703,"uuid":"202412608","full_name":"adobe/himl","owner":"adobe","description":"A hierarchical yaml config in Python","archived":false,"fork":false,"pushed_at":"2026-07-24T07:43:53.000Z","size":349,"stargazers_count":134,"open_issues_count":12,"forks_count":34,"subscribers_count":14,"default_branch":"main","last_synced_at":"2026-07-24T09:12:17.020Z","etag":null,"topics":["config","hierarchical","pypi-package","python","ssm","yaml"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/himl/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adobe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2019-08-14T19:23:44.000Z","updated_at":"2026-07-23T12:31:02.000Z","dependencies_parsed_at":"2026-02-13T06:01:19.987Z","dependency_job_id":"67576d02-5d13-4cb2-bb30-915f7bb71e82","html_url":"https://github.com/adobe/himl","commit_stats":{"total_commits":151,"total_committers":15,"mean_commits":"10.066666666666666","dds":0.7350993377483444,"last_synced_commit":"298793c94fb9f6e8d68af00f4a09f8b5c6ef4d24"},"previous_names":[],"tags_count":64,"template":false,"template_full_name":null,"purl":"pkg:github/adobe/himl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fhiml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fhiml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fhiml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fhiml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adobe","download_url":"https://codeload.github.com/adobe/himl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fhiml/sbom","scorecard":{"id":167280,"data":{"date":"2025-08-11","repo":{"name":"github.com/adobe/himl","commit":"ab55cd8821e96309f5277fc157f2477189552348"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":6.1,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":10,"reason":"25 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":3,"reason":"Found 2/6 approved changesets -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'contents' permission set to 'read': .github/workflows/publish-docker-image.yml:15","Warn: no topLevel permission defined: .github/workflows/publish-docker-image.yml:1","Warn: no topLevel permission defined: .github/workflows/publish-to-pypi.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish-docker-image.yml:20: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-docker-image.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-docker-image.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-docker-image.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-docker-image.yml:36: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-docker-image.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-docker-image.yml:43: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-docker-image.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish-to-pypi.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-to-pypi.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/publish-to-pypi.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-to-pypi.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/publish-to-pypi.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/adobe/himl/publish-to-pypi.yml/main?enable=pin","Warn: pipCommand not pinned by hash: Dockerfile:9","Warn: pipCommand not pinned by hash: Dockerfile:9","Warn: pipCommand not pinned by hash: Dockerfile:9","Warn: pipCommand not pinned by hash: .github/workflows/publish-to-pypi.yml:21","Warn: pipCommand not pinned by hash: .github/workflows/publish-to-pypi.yml:25","Warn: pipCommand not pinned by hash: .github/workflows/publish-to-pypi.yml:30","Info:   0 out of   3 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of   4 third-party GitHubAction dependencies pinned","Info:   1 out of   1 containerImage dependencies pinned","Info:   0 out of   6 pipCommand dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/publish-docker-image.yml:12"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Branch-Protection","score":-1,"reason":"internal error: error during branchesHandler.setup: internal error: githubv4.Query: Resource not accessible by integration","details":null,"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found linked content: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found text in security policy: github.com/adobe/.github/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 28 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T15:15:42.079Z","repository_id":37546703,"created_at":"2025-08-16T15:15:42.079Z","updated_at":"2025-08-16T15:15:42.079Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":36170392,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-08-01T02:00:05.789Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","hierarchical","pypi-package","python","ssm","yaml"],"created_at":"2026-08-01T21:01:56.386Z","updated_at":"2026-08-01T21:01:57.252Z","avatar_url":"https://github.com/adobe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# himl\n\n[![CI](https://github.com/adobe/himl/workflows/CI/badge.svg)](https://github.com/adobe/himl/actions)\n[![codecov](https://codecov.io/gh/adobe/himl/branch/main/graph/badge.svg)](https://codecov.io/gh/adobe/himl)\n[![PyPI version](https://badge.fury.io/py/himl.svg)](https://badge.fury.io/py/himl)\n[![Python versions](https://img.shields.io/pypi/pyversions/himl.svg)](https://pypi.org/project/himl/)\n\nA hierarchical config using yaml in Python.\n\nLatest version is: **0.20.0**\n\n\u003e **⚠️ Breaking Changes in v0.18.0**\n\u003e This version includes breaking changes:\n\u003e - Default list merge strategy changed from `append` to `append_unique`\n\u003e - Enhanced path validation (raises `FileNotFoundError` for non-existent paths)\n\u003e - Empty directory validation (raises `Exception` for directories without YAML files)\n\n## Description\n\nA python module which allows you to merge hierarchical config files using YAML syntax. It offers deep merge, variable interpolation and secrets retrieval from secrets managers.\n\nIt is ideal if you want to structure your hierarchy in such a way that you avoid duplication. You can define a structure for your configuration using a hierarchy such environment/project/cluster/app. It is up to you what layers you want to use in this hierarchy. The tool will read all yaml files starting from the root (where default values would be) all the way to the leaf (where most specific values would be, which will take precedence).\n\nIdea came from puppet's hiera.\n\n## Table of Contents\n\n- [himl](#himl)\n  - [Description](#description)\n  - [Table of Contents](#table-of-contents)\n  - [Installation](#installation)\n    - [Using `pip`](#using-pip)\n    - [Using `docker` image](#using-docker-image)\n    - [From Source](#from-source)\n  - [Examples](#examples)\n    - [Using the python module](#using-the-python-module)\n    - [Using the cli](#using-the-cli)\n  - [Features](#features)\n    - [Interpolation](#interpolation)\n      - [Interpolating simple values](#interpolating-simple-values)\n      - [Interpolating whole `dict`](#interpolating-whole-dict)\n    - [Deep merge](#deep-merge)\n    - [Secrets retrieval](#secrets-retrieval)\n      - [AWS SSM](#aws-ssm)\n      - [AWS S3](#aws-s3)\n      - [Vault](#vault)\n    - [Merge with Terraform remote state](#merge-with-terraform-remote-state)\n    - [Merge with env variables](#merge-with-env-variables)\n    - [Unicode Support](#unicode-support)\n  - [himl config merger](#himl-config-merger)\n    - [Output filtering](#output-filtering)\n    - [Extra merger features](#extra-merger-features)\n  - [Custom merge strategy](#custom-merge-strategy)\n  - [Development](#development)\n\n## Installation\n\n### Using `pip`\n\n```sh\npip install himl\n# or, optionally install with all extras dependencies: vault and s3 interpolation support\npip install himl[all]\n# or, specify which extras dependencies you need\npip install himl[vault,s3]\n```\n\n### Using `docker` image\n\n```sh\ndocker run ghcr.io/adobe/himl:latest himl-config-merger --help\n```\nSee all docker tags at: https://github.com/adobe/himl/pkgs/container/himl/versions\n\n### From Source\n\n```\ngit clone https://github.com/adobe/himl\ncd himl\nsudo python install -e .\n```\n\n## Examples\n\n### Using the python module\n\nThis will merge simple/default.yaml with simple/production/env.yaml\n```py\nfrom himl import ConfigProcessor\n\nconfig_processor = ConfigProcessor()\npath = \"examples/simple/production\"\nfilters = () # can choose to output only specific keys\nexclude_keys = () # can choose to remove specific keys\noutput_format = \"yaml\" # yaml/json\n\n\nconfig_processor.process(path=path, filters=filters, exclude_keys=exclude_keys,\n                         output_format=output_format, print_data=True)\n\n```\n\nThe above example will merge `simple/default.yaml` with `simple/production/env.yaml`:\n```\n$ tree examples/simple\nexamples/simple\n├── default.yaml\n└── production\n    └── env.yaml\n```\n\nThe example also showcases deep merging of lists and maps.\n\n`examples/simple/default.yaml`\n```yaml\n---\nenv: default\ndeep:\n  key1: v1\n  key2: v2\ndeep_list:\n  - item1\n  - item2\n```\n\n`examples/simple/production/env.yaml`\n```yaml\n---\nenv: prod\ndeep:\n  key3: v3\ndeep_list:\n  - item3\n```\n\nResult:\n```yaml\nenv: prod\ndeep:\n  key1: v1\n  key2: v2\n  key3: v3\ndeep_list:\n- item1\n- item2\n- item3\n```\n\n### Using the cli\n\nA cli tool called `himl` is automatically installed via `pip`. You can use it to parse a tree of yamls and it will either output the combined configuration at standard output or write it to a file.\n\n```sh\nusage: himl [-h] [--output-file OUTPUT_FILE] [--format OUTPUT_FORMAT]\n             [--filter FILTER] [--exclude EXCLUDE]\n             [--skip-interpolation-validation]\n             [--skip-interpolation-resolving] [--enclosing-key ENCLOSING_KEY]\n             [--cwd CWD] [--multi-line-string]\n             [--list-merge-strategy {append,override,prepend,append_unique}]\n             [--allow-unicode]\n             path\n```\n\n```sh\nhiml examples/complex/env=dev/region=us-east-1/cluster=cluster2\n```\n\nBased on the configuration tree from the [examples/complex](examples/complex) folder, the output of the above command will be the following:\n```\ncluster:\n  description: 'This is cluster: cluster2. It is using c3.2xlarge instance type.'\n  name: cluster2\n  node_type: c3.2xlarge\nregion:\n  location: us-east-1\nenv: dev\n```\n\nWhere the examples folder looks something like this:\n```\n$ tree examples/complex\nexamples/complex\n├── default.yaml\n├── env=dev\n│   ├── env.yaml\n│   ├── region=us-east-1\n│   │   ├── cluster=cluster1\n│   │   │   └── cluster.yaml\n│   │   ├── cluster=cluster2\n│   │   │   └── cluster.yaml\n│   │   └── region.yaml\n│   └── region=us-west-2\n│       ├── cluster=cluster1\n│       │   └── cluster.yaml\n│       └── region.yaml\n└── env=prod\n    ├── env.yaml\n    └── region=eu-west-2\n        ├── cluster=ireland1\n        │   └── cluster.yaml\n        └── region.yaml\n```\n\n## Features\n\n### Interpolation\n\nIn order to avoid repetition, we wanted to make it possible to define a value once and reuse it in other parts of the yaml config.\nUnlike yaml anchors, these interpolations work across multiple files.\n\n#### Interpolating simple values\n\n`data/default.yaml`:\n```yaml\nallowed_roles:\n  - \"arn:aws:iam::{{account.id}}:role/myrole\"\n```\n\n`data/dev/env.yaml`:\n```\naccount:\n  id: \"123456\"\n```\n\n#### Interpolating whole `dict`\n\n```yaml\nprojects:\n  webapp1:\n    tagging:\n      Owner: \"Web Service Team\"\n      Environment: \"dev\"\n      CostCenter: \"123\"\n  data-store:\n      Owner: \"Backend Team\"\n      Environment: \"dev\"\n      CostCenter: \"455\"\n\n# this will copy the whole projects.webapp1.tagging dict to this key\ntagging: \"{{projects.webapp1.tagging}}\"\n\n# or even a double interpolation\ntagging: \"{{projects.{{project.name}}.tagging}}\"\n```\n\n### Deep merge\n\nIt's possible to have the same key (eg. a dict/list) in multiple files and combine them using a deep merge.\nSee an example [here](https://github.com/adobe/himl#deep-merge-example).\n\n### Secrets retrieval\n\n#### [AWS SSM](https://docs.aws.amazon.com/systems-manager/latest/userguide/integration-ps-secretsmanager.html)\n\n```yaml\npassphrase: \"{{ssm.path(/key/coming/from/aws/secrets/store/manager).aws_profile(myprofile)}}\"\n```\n\n#### [AWS S3](https://aws.amazon.com/s3/)\n\n```yaml\nmy_value: \"{{s3.bucket(my-bucket).path(path/to/file.txt).base64encode(true).aws_profile(myprofile)}}\"\n```\n\n#### [Vault](https://www.vaultproject.io/)\n\nUse vault cli to authenticate, fallback method via LDAP.\n\nRetrieve only one key value from a secret, the path tail is used as key:\n```yaml\nmy_value: \"{{vault.key(/path/from/vault/key)}}\"\n```\n\nRetrieve all key/value pairs from a vault path:\n```yaml\nmy_dict: \"{{vault.path(/path/from/vault)}}\"\n```\n\nGenerate a token for a policy:\n```yaml\nmy_token: \"{{vault.token_policy(my_vault_policy)}}\"\n```\n\n### Merge with [Terraform remote state](https://www.terraform.io/docs/state/remote.html)\n\n```yaml\n### Terraform remote states ###\nremote_states:\n  - name: cluster_composition\n    type: terraform\n    aws_profile: \"my_aws_profile\"\n    s3_bucket: \"my_terraform_bucket\"\n    s3_key: \"mycluster.tfstate\"\n\n\nendpoint: \"{{outputs.cluster_composition.output.value.redis_endpoint}}\"\n```\n\n### Merge with env variables\n```yaml\nkubeconfig_location: \"{{env(KUBECONFIG)}}\"\n```\n\n### Unicode Support\n\nhiml supports Unicode characters in configuration files, allowing you to use international languages, special characters, and emoji in your YAML configs.\n\nBy default, Unicode characters are escaped in the output to ensure compatibility. You can preserve Unicode characters in their original form using the `--allow-unicode` flag.\n\n**Using the CLI:**\n```sh\n# With Unicode escaping (default)\nhiml examples/simple/production --output-file config.yaml\n\n# Preserving Unicode characters\nhiml examples/simple/production --output-file config.yaml --allow-unicode\n```\n\n**Using the Python module:**\n```py\nfrom himl import ConfigProcessor\n\nconfig_processor = ConfigProcessor()\npath = \"examples/simple/production\"\n\n# Process with Unicode preservation\nconfig = config_processor.process(\n    path=path,\n    output_format=\"yaml\",\n    allow_unicode=True,  # Preserve Unicode characters\n    print_data=True\n)\n```\n\n**Example with Unicode content:**\n\n`config/default.yaml`:\n```yaml\nservice:\n  name: \"My Service\"\n  description: \"Multi-language support: English, 中文, العربية, Русский\"\n\nmessages:\n  welcome:\n    en: \"Welcome\"\n    zh: \"欢迎\"\n    ar: \"مرحبا\"\n    ru: \"Добро пожаловать\"\n\nteam:\n  - name: \"José García\"\n    role: \"Developer\"\n  - name: \"田中太郎\"\n    role: \"Designer\"\n```\n\nWhen processed with `--allow-unicode`, the output preserves all Unicode characters. Without the flag, non-ASCII characters are escaped (e.g., `\\u4e2d\\u6587` for Chinese characters).\n\n**Note:** Some emoji and 4-byte UTF-8 characters may be escaped by the YAML library even with `--allow-unicode` enabled.\n\n\n## himl-config-merger\n\nThe `himl-config-merger` script, contains logic of merging a hierarchical config directory and creating the end result YAML files.\n\n```sh\nhiml-config-merger examples/complex --output-dir merged_output --levels env region cluster --leaf-directories cluster\n```\n\n```\nINFO:__main__:Found input config directory: examples/complex/env=prod/region=eu-west-2/cluster=ireland1\nINFO:__main__:Storing generated config to: merged_output/prod/eu-west-2/ireland1.yaml\nINFO:__main__:Found input config directory: examples/complex/env=dev/region=us-west-2/cluster=cluster1\nINFO:__main__:Storing generated config to: merged_output/dev/us-west-2/cluster1.yaml\nINFO:__main__:Found input config directory: examples/complex/env=dev/region=us-east-1/cluster=cluster1\nINFO:__main__:Storing generated config to: merged_output/dev/us-east-1/cluster1.yaml\nINFO:__main__:Found input config directory: examples/complex/env=dev/region=us-east-1/cluster=cluster2\nINFO:__main__:Storing generated config to: merged_output/dev/us-east-1/cluster2.yaml\n```\n\nInput example:\n```\n\u003e tree examples/complex\nexamples/complex\n├── default.yaml\n├── env=dev\n│   ├── env.yaml\n│   ├── region=us-east-1\n│   │   ├── cluster=cluster1\n│   │   │   └── cluster.yaml\n│   │   ├── cluster=cluster2\n│   │   │   └── cluster.yaml\n│   │   └── region.yaml\n│   └── region=us-west-2\n│       ├── cluster=cluster1\n│       │   └── cluster.yaml\n│       └── region.yaml\n└── env=prod\n    ├── env.yaml\n    └── region=eu-west-2\n        ├── cluster=ireland1\n        │   └── cluster.yaml\n        └── region.yaml\n```\n\nOutput:\n```\nmerged_output\n├── dev\n│   ├── us-east-1\n│   │   ├── cluster1.yaml\n│   │   └── cluster2.yaml\n│   └── us-west-2\n│       └── cluster1.yaml\n└── prod\n    └── eu-west-2\n        └── ireland1.yaml\n```\n\nLeveraging HIML, the config-merger script loads the configs tree structure and deep-merges all keys from all YAML files found from a root path to an edge. For each leaf directory, a file will be created under `--output-dir`.\n\nUnder each level, there is a mandatory \"level key\" that is used by config-merger for computing the end result. This key should be present in one of the files under each level. (eg. env.yaml under env).\n\n### Output filtering\n\nSome configs that are specified in the higher levels of the directory tree might not be needed in the end (leaf) result. For this reason, the config-merger script can apply a set of filter rules that are specified via the `--filter-rules-key` parameter. This property must be present in the config and contains rules for removing root level keys from the output. The filter is applied if the selector object matches a subset of the output keys and will keep the keys specified in the `values` list or the keys that match the `regex` pattern.\n\n\n```yaml\n# intermediate config after hierarchical merge\nenv: dev\ncluster: cluster1\nregion: us-east-1\nkey1: persisted\nkey2: dropped\nkeep_1: persisted\ntags:\n  cost_center: 123\n_filters:\n- selector:\n    env: \"dev\"\n  keys:\n    values:\n    - key1\n    regex: \"keep_.*\"\n- selector:\n    cluster:\n      regex: \"cluster1\"\n  keys:\n    values:\n    - tags\n```\n\nBuild the output with filtering:\n```sh\nhiml-config-merger examples/filters --output-dir merged_output --levels env region cluster --leaf-directories cluster --filter-rules-key _filters\n```\n\nThe `himl-config-merger` command also supports the `--allow-unicode` flag for preserving Unicode characters in the merged output files:\n\n```sh\nhiml-config-merger examples/complex --output-dir merged_output --levels env region cluster --leaf-directories cluster --allow-unicode\n```\n\n```yaml\n# output after filtering\nenv: dev\ncluster: cluster1\nregion: us-east-1\nkey1: persisted\nkeep_1: persisted\ntags:\n  cost_center: 123\n```\n#### Filtering limitations\n\nRule selectors and keys filtering only works at the root level of the config. It is not possible to filter nested keys.\n\n\n### Extra merger features\n\nApart from the standard features found in the `PyYaml` library, the `himl-config-merger` component also implements a custom YAML tag called `!include`.\n\nExample:\n```yaml\nVA7:     !include configs/env=int/region=va7/kafka-brokers.yaml regionBrokers.VA7\n```\n\nThis will replace the value after interpolation with the value of the regionBrokers.VA7 found under the configs/env=int/region=va7/kafka-brokers.yaml path.\n\n## Custom merge strategy\nAn optional parameter `type_strategies` can be passed into ConfigProcessor to define custom merging behavior. It could be custom functions that fit your needs.\nYour function should take the arguments of (config, path, base, nxt) and return the merged result.\n\nExample:\n```py\nfrom himl import ConfigProcessor\n\ndef strategy_merge_override(config, path, base, nxt):\n    \"\"\"merge list of dicts. if objects have same id, nxt replaces base.\"\"\"\n    \"\"\"if remove flag is present in nxt item, remove base and not add nxt\"\"\"\n    result = deepcopy(base)\n    for nxto in nxt:\n        for baseo in result:\n            # if list is not a list of dicts, bail out and let the next strategy to execute\n            if not isinstance(baseo,dict) or not isinstance(nxto,dict):\n                return STRATEGY_END\n            if 'id' in baseo and 'id' in nxto and baseo['id'] == nxto['id']:\n                result.remove(baseo) #same id, remove previous item\n        if 'remove' not in nxto:\n            result.append(nxto)\n    return result\n\nconfig_processor = ConfigProcessor()\npath = \"examples/simple/production\"\nfilters = () # can choose to output only specific keys\nexclude_keys = () # can choose to remove specific keys\noutput_format = \"yaml\" # yaml/json\n\nconfig_processor.process(path=path, filters=filters, exclude_keys=exclude_keys,\n                         output_format=output_format, print_data=True,\n                         type_strategies= [(list, [strategy_merge_override,'append']), (dict, [\"merge\"])] ))\n\n```\n\n## Development\n\n### Setting up development environment\n\n1. Clone the repository:\n```bash\ngit clone https://github.com/adobe/himl.git\ncd himl\n```\n\n2. Install in development mode with all dependencies:\n```bash\npip install -e .[dev]\n```\n\n3. Run tests:\n```bash\npytest tests/\n```\n\n4. Run tests with coverage:\n```bash\npytest tests/ --cov=himl --cov-report=term-missing\n```\n\n5. Run linting:\n```bash\nflake8 .\n```\n\n### CI/CD\n\nThis project uses GitHub Actions for continuous integration. The CI pipeline:\n\n- **Tests**: Runs on Python 3.10-3.14 across Ubuntu and macOS\n- **Security**: Runs bandit security checks and safety dependency checks\n- **Build**: Builds and validates the package\n- **Integration**: Tests CLI tools and package installation\n- **Coverage**: Reports test coverage to Codecov\n\nAll tests must pass and maintain good test coverage before merging PRs.\n\n### Version Management\n\nThis project uses [bump-my-version](https://github.com/callowayproject/bump-my-version) for version management. The version is automatically determined from Git tags using setuptools_scm.\n\n#### Bumping Versions\n\nUse the provided Makefile commands:\n\n```bash\n# Show what version bumps would result in\nmake show-bump\n\n# Dry run version bumps (to see what would change)\nmake dry-bump-patch   # 0.16.4 -\u003e 0.16.5\nmake dry-bump-minor   # 0.16.4 -\u003e 0.17.0\nmake dry-bump-major   # 0.16.4 -\u003e 1.0.0\n\n# Actually bump the version\nmake bump-patch       # For bug fixes\nmake bump-minor       # For new features\nmake bump-major       # For breaking changes\n```\n\nThe version bump will:\n1. Update the version in `pyproject.toml`\n2. Update the version in `README.md`\n3. Update the version in `himl/main.py` (CLI help)\n4. Create a Git commit with the changes\n5. Create a Git tag with the new version\n\n### Contributing\n\n1. Fork the repository\n2. Create a feature branch (`git checkout -b feature/amazing-feature`)\n3. Make your changes and add tests\n4. Ensure all tests pass (`pytest tests/`)\n5. Commit your changes (`git commit -m 'Add amazing feature'`)\n6. Push to the branch (`git push origin feature/amazing-feature`)\n7. Open a Pull Request\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fhiml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadobe%2Fhiml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fhiml/lists"}