{"id":13820930,"url":"https://github.com/zerwes/hiyapyco","last_synced_at":"2025-04-05T08:08:09.594Z","repository":{"id":23126330,"uuid":"26481039","full_name":"zerwes/hiyapyco","owner":"zerwes","description":"HiYaPyCo - A Hierarchical Yaml Python Config","archived":false,"fork":false,"pushed_at":"2024-05-12T05:09:04.000Z","size":302,"stargazers_count":102,"open_issues_count":4,"forks_count":22,"subscribers_count":9,"default_branch":"main","last_synced_at":"2024-05-14T12:12:22.171Z","etag":null,"topics":["config","configuration","merge","open-source","python","python3","yaml"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zerwes.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-11-11T10:53:40.000Z","updated_at":"2024-05-29T21:50:08.975Z","dependencies_parsed_at":"2024-05-29T21:50:06.724Z","dependency_job_id":"bd2d29d1-4c9e-49dd-8638-9bbc8ccb8c01","html_url":"https://github.com/zerwes/hiyapyco","commit_stats":{"total_commits":311,"total_committers":15,"mean_commits":"20.733333333333334","dds":"0.16720257234726688","last_synced_commit":"64dbaa65e81b497718707027048bc6f9a67b6d6e"},"previous_names":[],"tags_count":32,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerwes%2Fhiyapyco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerwes%2Fhiyapyco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerwes%2Fhiyapyco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zerwes%2Fhiyapyco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zerwes","download_url":"https://codeload.github.com/zerwes/hiyapyco/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247305934,"owners_count":20917208,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","configuration","merge","open-source","python","python3","yaml"],"created_at":"2024-08-04T08:01:11.860Z","updated_at":"2025-04-05T08:08:09.573Z","avatar_url":"https://github.com/zerwes.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":".. |pylint| image:: https://github.com/zerwes/hiyapyco/actions/workflows/pylint.yml/badge.svg?branch=main\n    :target: https://github.com/zerwes/hiyapyco/actions/workflows/pylint.yml\n.. |test| image:: https://github.com/zerwes/hiyapyco/actions/workflows/test.yml/badge.svg\n     :target: https://github.com/zerwes/hiyapyco/actions/workflows/test.yml\n.. |gpl| image:: https://img.shields.io/badge/License-GPL%20v3-blue.svg\n     :target: http://www.gnu.org/licenses/gpl-3.0\n\n|pylint| |test| |gpl|\n\nhiyapyco\n========\n\nHiYaPyCo - A Hierarchical Yaml Python Config\n\nDescription\n-----------\n\nA simple python lib allowing hierarchical overlay of config files in\nYAML syntax, offering different merge methods and variable interpolation\nbased on jinja2.\n\nThe goal was to have something similar to puppets hiera\n``merge_behavior: deeper`` for python.\n\nKey Features\n------------\n\n-  hierarchical overlay of multiple YAML files\n-  multiple merge methods for hierarchical YAML files\n-  variable interpolation using jinja2\n\nRequirements\n------------\n\n-  PyYAML aka. python3-yaml\n-  Jinja2 aka. python3-jinja2\n\nPython Version\n~~~~~~~~~~~~~~\n\nHiYaPyCo was designed to run on current major python versions\nwithout changes. Tested versions:\n\n-  3.9\n-  3.11\n\nUsage\n-----\n\nA simple example:\n\n::\n\n    import hiyapyco\n    conf = hiyapyco.load('yamlfile1' [,'yamlfile2' [,'yamlfile3' [...]]] [,kwargs])\n    print(hiyapyco.dump(conf, default_flow_style=False))\n\nreal life example:\n~~~~~~~~~~~~~~~~~~\n\n``yaml1.yaml``:\n\n::\n\n    ---\n    first: first element\n    second: xxx\n    deep:\n        k1:\n            - 1\n            - 2\n\n``yaml2.yaml``:\n\n::\n\n    ---\n    second: again {{ first }}\n    deep:\n        k1:\n            - 4 \n            - 6\n        k2:\n            - 3\n            - 6\n\nload ...\n\n::\n\n    \u003e\u003e\u003e import pprint\n    \u003e\u003e\u003e import hiyapyco\n    \u003e\u003e\u003e conf = hiyapyco.load('yaml1.yaml', 'yaml2.yaml', method=hiyapyco.METHOD_MERGE, interpolate=True, failonmissingfiles=True)\n    \u003e\u003e\u003e pprint.PrettyPrinter(indent=4).pprint(conf)\n    {   'deep': {   'k1': [1, 2, 4, 6], 'k2': [3, 6]},\n        'first': u'first element',\n        'ma': {   'ones': u'12', 'sum': u'22'},\n        'second': u'again first element'}\n\nreal life example using yaml documents as strings\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\n::\n\n    \u003e\u003e\u003e import hiyapyco\n    \u003e\u003e\u003e y1=\"\"\"\n    ... yaml: 1\n    ... y:\n    ...   y1: abc\n    ...   y2: xyz\n    ... \"\"\"\n    \u003e\u003e\u003e y2=\"\"\"\n    ... yaml: 2\n    ... y:\n    ...   y2: def\n    ...   y3: XYZ\n    ... \"\"\"\n    \u003e\u003e\u003e conf = hiyapyco.load([y1, y2], method=hiyapyco.METHOD_MERGE)\n    \u003e\u003e\u003e print (conf)\n    OrderedDict([('yaml', 2), ('y', OrderedDict([('y1', 'abc'), ('y2', 'def'), ('y3', 'XYZ')]))])\n    \u003e\u003e\u003e hiyapyco.dump(conf, default_flow_style=True)\n    '{yaml: 2, y: {y1: abc, y2: def, y3: XYZ}}\\n'\n\nargs\n~~~~\n\nAll ``args`` are handled as *file names* or *yaml documents*. They may\nbe strings or list of strings.\n\nkwargs\n~~~~~~\n\n-  ``method``: bit (one of the listed below):\n\n   -  ``hiyapyco.METHOD_SIMPLE``: replace values (except for lists a\n      simple merge is performed) (default method)\n   -  ``hiyapyco.METHOD_MERGE``: perform a deep merge\n   -  ``hiyapyco.METHOD_SUBSTITUTE``: perform a merge w/ lists substituted (unsupported)\n\n-  ``mergelists``: boolean try to merge lists of dict (default: ``True``)\n\n-  ``none_behavior``: bit (one of the listed below):\n\n   -  ``hiyapyco.NONE_BEHAVIOR_DEFAULT``: attempt to merge the value with ``None`` and fail if this is not possible (default method)\n   -  ``hiyapyco.NONE_BEHAVIOR_OVERRIDE``: ``None`` always overrides any other value.\n\n-  ``interpolate``: boolean : perform interpolation after the merge\n   (default: ``False``)\n\n-  ``castinterpolated``: boolean : try to perform a *best possible\n   match* cast for interpolated strings (default: ``False``)\n\n-  ``usedefaultyamlloader``: boolean : force the usage of the default\n   *PyYAML* loader/dumper instead of *HiYaPyCo*\\ s implementation of a\n   OrderedDict loader/dumper (see: Ordered Dict Yaml Loader / Dumper\n   aka. ODYLDo) (default: ``False``)\n\n-  ``dereferenceyamlanchors``: boolean : dereference yaml anchors and use a copy (default: ``True``)\n\n-  ``encoding``: string : encoding used to read yaml files (default: ``utf-8``)\n\n-  ``failonmissingfiles``: boolean : fail if a supplied YAML file can\n   not be found (default: ``True``)\n\n-  ``loglevel``: int : loglevel for the hiyapyco logger; should be one\n   of the valid levels from ``logging``: 'WARN', 'ERROR', 'DEBUG', 'I\n   NFO', 'WARNING', 'CRITICAL', 'NOTSET' (default: default of\n   ``logging``)\n\n-  ``loglevelmissingfiles``: int : one of the valid levels from\n   ``logging``: 'WARN', 'ERROR', 'DEBUG', 'INFO', 'WARNING', 'CRITICAL',\n   'NOTSET' (default: ``logging.ERROR`` if\n   ``failonmissingfiles = True``, else ``logging.WARN``)\n\n-  ``mergeoverride``: optional function to customize merge for primitive values\n   (see `PR #76 \u003chttps://github.com/zerwes/hiyapyco/pull/76\u003e`_.)\n\n-  ``loader_callback``: optional custom callback function to load yaml files.\n   The callback function shall behave like ``yaml.load_all`` from PyYAML,\n   taking a IO stream as input and returning a list of objects.\n   Using this method, for example `ruamel \u003chttps://pypi.org/project/ruamel.yaml/\u003e`_\n   can be used instead of PyYAML etc.\n\ninterpolation\n~~~~~~~~~~~~~\n\nFor using interpolation, I strongly recomend *not* to use the default\nPyYAML loader, as it sorts the dict entrys alphabetically, a fact that\nmay break interpolation in some cases (see ``test/odict.yaml`` and\n``test/test_odict.py`` for an example). See Ordered Dict Yaml Loader /\nDumper aka. ODYLDo\n\ndefault\n^^^^^^^\n\nThe default jinja2.Environment for the interpolation is\n\n::\n\n    hiyapyco.jinja2env = Environment(undefined=Undefined)\n\nThis means that undefined vars will be ignored and replaced with a empty\nstring.\n\nchange the jinja2 Environment\n^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf you like to change the jinja2 Environment used for the interpolation,\nset ``hiyapyco.jinja2env`` **before** calling ``hiyapyco.load``!\n\nuse jinja2 DebugUndefined\n^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf you like to keep the undefined var as string but raise no error, use\n\n::\n\n    from jinja2 import Environment, Undefined, DebugUndefined, StrictUndefined\n    hiyapyco.jinja2env = Environment(undefined=DebugUndefined)\n\nuse jinja2 StrictUndefined\n^^^^^^^^^^^^^^^^^^^^^^^^^^\n\nIf you like to raise a error on undefined vars, use\n\n::\n\n    from jinja2 import Environment, Undefined, DebugUndefined, StrictUndefined\n    hiyapyco.jinja2env = Environment(undefined=StrictUndefined)\n\nThis will raise a ``hiyapyco.HiYaPyCoImplementationException`` wrapped\narround the ``jinja2.UndefinedError`` pointing at the string causing the\nerror.\n\nmore informations\n^^^^^^^^^^^^^^^^^\n\nSee:\n`jinja2.Environment \u003chttp://jinja.pocoo.org/docs/dev/api/#jinja2.Environment\u003e`_\n\ncast interpolated strings\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nAs you must use interpolation as strings (PyYAML will weep if you try to\nstart a value with ``{{``), you can set ``castinterpolated`` to *True*\nin order to try to get a ``best match`` cast for the interpolated\nvalues. **The ``best match`` cast is currently only a q\u0026d implementation\nand may not give you the expected results!**\n\nOrdered Dict Yaml Loader / Dumper aka. ODYLDo\n~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nThis is a simple implementation of a PyYAML loader / dumper using\n``OrderedDict`` from collections.\n**Because chaos is fun but order matters on loading dicts from a yaml\nfile.**\n\n\nInstall\n-------\n\nFrom Source\n~~~~~~~~~~~\n\nGitHub\n^^^^^^\n\n`https://github.com/zerwes/hiyapyco \u003chttps://github.com/zerwes/hiyapyco\u003e`_\n\n::\n\n    git clone https://github.com/zerwes/hiyapyco\n    cd hiyapyco\n    sudo python setup.py install\n\nPyPi\n^^^^\n\nDownload the latest or desired version of the source package from\n`https://pypi.python.org/pypi/HiYaPyCo \u003chttps://pypi.python.org/pypi/HiYaPyCo\u003e`_.\nUnpack the archive and install by executing:\n\n::\n\n    sudo python setup.py install\n\npip\n~~~\n\nInstall the latest wheel package using:\n\n::\n\n    pip install HiYaPyCo\n\ndebian packages\n~~~~~~~~~~~~~~~\n\ninstall the latest debian packages from http://repo.zero-sys.net/hiyapyco::\n\n    # create the sources list file:\n    sudo echo \"deb http://repo.zero-sys.net/hiyapyco/deb ./\" \u003e /etc/apt/sources.list.d/hiyapyco.list\n\n    # import the key:\n    gpg --keyserver keys.gnupg.net --recv-key 77DE7FB4\n    # or use:\n    wget https://repo.zero-sys.net/77DE7FB4.asc -O - | gpg --import -\n\n    # apt tasks:\n    gpg --armor --export 77DE7FB4 | sudo tee /etc/apt/trusted.gpg.d/hiyapyco.asc\n    sudo apt-get update\n    sudo apt-get install python3-hiyapyco\n\na ansible playbook exists: https://github.com/zerwes/ansible-role-hiyapyco\n\nrpm packages\n~~~~~~~~~~~~\n\nuse\n`http://repo.zero-sys.net/hiyapyco/rpm \u003chttp://repo.zero-sys.net/hiyapyco/rpm\u003e`_\nas URL for the yum repo and\n`https://repo.zero-sys.net/77DE7FB4.asc \u003chttps://repo.zero-sys.net/77DE7FB4.asc\u003e`_\nas the URL for the key.\n\nArch Linux\n~~~~~~~~~~\n\nAn `AUR package \u003chttps://aur.archlinux.org/packages/python-hiyapyco/\u003e`_\nis available (provided by `Pete Crighton \u003chttps://github.com/PeteCrighton\u003e`_ and not always up to date).\n\nLicense\n-------\n\nCopyright |copy| 2014 - 2024 Klaus Zerwes `zero-sys.net \u003chttps://zero-sys.net\u003e`_\n\n.. |copy| unicode:: 0xA9 .. copyright sign\n\nThis package is free software.\nThis software is licensed under the terms of the GNU GENERAL PUBLIC\nLICENSE version 3 or later, as published by the Free Software\nFoundation.\nSee\n`https://www.gnu.org/licenses/gpl.html \u003chttps://www.gnu.org/licenses/gpl.html\u003e`_\n\nChangelog\n---------\n\n0.7.0\n~~~~~~\n\nMERGED: allow custom yaml loaders as callback functions by @grst (PR #77)\n\nMERGED: implement none-behavior strategies by @grst (PR #78)\n\nMERGED: update markupsafe requirement from \u003c3 to \u003c4 (#80)\n\nIMPROVED: added some example how to use ruamel\n\n0.6.1\n~~~~~~\n\nMERGED: #76 Override mechanism for primitive value merge by malachib\n\nIMPROVED: added link to ansible playbook\n\n0.6.0\n~~~~~~\n\nFIXED: #69 (weird merge behavior with anchors)\n\nMERGED: #71 (dereference anchors)\n\n0.5.6\n~~~~~~\n\nMERGED: #70 by itachi-cracker\n\nFIXED: #61 (removed deprecated distutils)\n\n0.5.5\n~~~~~~\n\nFIXED: #67 cosmetic changes\n\n0.5.4\n~~~~~~\n\nFIXED: #60 recursive calls to _substmerge\n\nIMPROVED: testing and python support (3.11)\n\n0.5.1\n~~~~~~\n\nMERGED: #52 by ryanfaircloth\n\n0.5.0\n~~~~~~\n\nMERGED: #41 Jinja2 dependency increased to include Jinja2 3.x.x\n\nREMOVED: Support for Python 2\n\n0.4.16\n~~~~~~\n\nMERGED: #37 alex-ber\n\n0.4.15\n~~~~~~\n\nMERGED: #30 lesiak:issue-30-utf\n\nMERGED: #28 lesiak:issue-28\n\n0.4.14\n~~~~~~\n\nFIXED: issue #33\n\nMERGED: issue #32\n\n0.4.13\n~~~~~~\n\nIMPLEMENTED: [issue #27] support multiple yaml documents in one file\n\n0.4.12\n~~~~~~\n\nFIXED: logging by Regev Golan\n\n0.4.11\n~~~~~~\n\nIMPLEMENTED: mergelists (see issue #25)\n\n0.4.10\n~~~~~~\n\nFIXED: issue #24 repo signing\n\n0.4.9\n~~~~~\n\nFIXED: issue #23 loglevelonmissingfiles\n\n0.4.8\n~~~~~\n\nFixed pypi doc\n\n0.4.7\n~~~~~\n\nReverted: logger settings to initial state\n\nImproved: dump\n\nMerged:\n\n- flatten mapping from Chris Petersen geek@ex-nerd.com\n- arch linux package info from Peter Crighton git@petercrighton.de\n\n0.4.6\n~~~~~\n\nMERGED: fixes from mmariani\n\n0.4.5\n~~~~~\n\nFIXED: issues #9 and #11\n\n0.4.4\n~~~~~\n\ndeb packages:\n\n- removed support for python 2.6\n- include examples as doc\n\n0.4.3\n~~~~~\n\nFIXED: issue #6 *import of hiyapyco **version** in setup.py causes pip\ninstall failures*\n\n0.4.2\n~~~~~\n\nChanged: moved to GPL\n\nImprovements: missing files handling, doc\n\n0.4.1\n~~~~~\n\nImplemented: ``castinterpolated``\n\n0.4.0\n~~~~~\n\nImplemented: loading yaml docs from string\n\n0.3.2\n~~~~~\n\nImproved tests and bool args checks\n\n0.3.0 / 0.3.1\n~~~~~~~~~~~~~\n\nImplemented a Ordered Dict Yaml Loader\n\n0.2.0\n~~~~~\n\nFixed unicode handling\n\n0.1.0 / 0.1.1\n~~~~~~~~~~~~~\n\nInitial release\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerwes%2Fhiyapyco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzerwes%2Fhiyapyco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzerwes%2Fhiyapyco/lists"}