{"id":15532888,"url":"https://github.com/matham/tree-config","last_synced_at":"2025-04-09T10:14:49.844Z","repository":{"id":57477040,"uuid":"275878772","full_name":"matham/tree-config","owner":"matham","description":"Configuration of objects that are nested in a tree-like fashion.","archived":false,"fork":false,"pushed_at":"2022-02-11T21:27:35.000Z","size":2982,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-15T17:47:55.475Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/matham.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-06-29T17:09:40.000Z","updated_at":"2022-02-10T22:42:59.000Z","dependencies_parsed_at":"2022-09-14T17:11:07.482Z","dependency_job_id":null,"html_url":"https://github.com/matham/tree-config","commit_stats":null,"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matham%2Ftree-config","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matham%2Ftree-config/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matham%2Ftree-config/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/matham%2Ftree-config/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/matham","download_url":"https://codeload.github.com/matham/tree-config/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248018070,"owners_count":21034048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-02T11:33:43.193Z","updated_at":"2025-04-09T10:14:49.816Z","avatar_url":"https://github.com/matham.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"tree-config\n===========\n\nConfiguration of objects that are nested in a tree-like fashion.\n\nFor more information: https://matham.github.io/tree-config/index.html\n\n.. image:: https://img.shields.io/pypi/pyversions/tree-config.svg\n    :target: https://pypi.python.org/pypi/tree-config/\n    :alt: Supported Python versions\n\n.. image:: https://img.shields.io/pypi/v/tree-config.svg\n    :target: https://pypi.python.org/pypi/tree-config/\n    :alt: Latest Version on PyPI\n\n.. image:: https://coveralls.io/repos/github/matham/tree-config/badge.svg?branch=main\n    :target: https://coveralls.io/github/matham/tree-config?branch=main\n    :alt: Coverage status\n\n.. image:: https://github.com/matham/tree-config/workflows/Python%20application/badge.svg\n    :target: https://github.com/matham/tree-config/actions\n    :alt: Github action status\n\nInstallation\n============\n\n``tree-config`` can be installed with:\n\n.. code-block:: shell\n\n    pip install tree-config\n\nConfiguration usage\n===================\n\n``tree-config`` can dump all the configurable properties of all your classes to\na yaml file and then load the the file and restore/apply the values to the\nproperties. E.g.:\n\n.. code-block:: python\n\n    class App:\n\n        _config_props_ = ('name', )\n\n        _config_children_ = {'app panel': 'panel'}\n\n        def __init__(self):\n            self.name = 'Desk'\n            self.panel = AppPanel()\n\n    class AppPanel:\n\n        _config_props_ = ('color', )\n\n        color = 'A4FF67'\n\nwill automatically configure ``name`` and ``color``.\n\nFollowing is a guide by example of the multiples ways to control the configuration.\nSee the `configuration API \u003chttps://matham.github.io/tree-config/api.html\u003e`_, including\nthe ``Configuration`` class for complete details.\n\nSee the examples directory in the repo for complete runnable code of the following\nexamples.\n\nBasic properties\n----------------\n\nThis example has an app class that contains two panels that are configurable.\n``_config_props_`` lists the configurable properties for the class, while\n``_config_children_`` constructs the tree of objects that are configurable.\n\n.. code-block:: python\n\n    class App:\n\n        _config_props_ = ('size', 'name')\n\n        _config_children_ = {'app panel': 'panel1', 'home panel': 'panel2'}\n\n        def __init__(self):\n            self.size = 12\n            self.name = 'Desk'\n\n            self.panel1 = AppPanel()\n            self.panel2 = HomePanel()\n\n\n    class AppPanel:\n\n        _config_props_ = ('color', )\n\n        color = 'A4FF67'\n\n\n    class HomePanel:\n\n        _config_props_ = ('shape', )\n\n        shape = 'circle'\n\nThen, running:\n\n.. code-block:: python\n\n    from tree_config import dump_config, read_config_from_object\n    app = App()\n    dump_config('basic_example.yaml', read_config_from_object(app))\n    print(f'Shape is: {app.panel2.shape}')\n\ncreates a ``basic_example.yaml`` file with the following contents:\n\n.. code-block:: yaml\n\n    app panel: {color: A4FF67}\n    home panel: {shape: circle}\n    name: Desk\n    size: 12\n\nand it prints ``Shape is: circle``. If we want to load a previous yaml file,\nwhere say the shape was ``\"square\"`` and apply it to the instance, we do:\n\n.. code-block:: python\n\n    from tree_config import load_config, apply_config\n    app = App()\n    apply_config(app, load_config(app, 'basic_example.yaml'))\n    print(f'Shape is: {app.panel2.shape}')\n\nThis in turn prints ``Shape is: square``.\n\nHooking property discovery\n--------------------------\n\n``_config_props_`` and ``_config_children_`` are defined on a class, not on\ninstances. When ``tree-config`` uses them, it will walk the whole class\nhierarchy and accumulate their values from all super classes because a\nsub-class does not overwrite them, but rather adds to them.\n\nIf ``_config_props`` and/or ``_config_children`` is defined on a\nclass or instance, tree-config will use that value directly, instead of\nwalking ``_config_props_`` and/or ``_config_children_``, respectively.\n\nE.g. the following code:\n\n.. code-block:: python\n\n    from tree_config import dump_config, read_config_from_object\n\n\n    class App:\n\n        _config_children_ = {'app panel': 'panel1', 'home panel': 'panel2'}\n\n        def __init__(self):\n            self.panel1 = AppPanel()\n            self.panel2 = HomePanel()\n\n\n    class RootPanel:\n\n        _config_props_ = ('size', 'name')\n\n        size = 12\n\n        name = 'Desk'\n\n\n    class AppPanel(RootPanel):\n\n        _config_props_ = ('color', )\n\n        color = 'A4FF67'\n\n\n    class HomePanel(AppPanel):\n\n        _config_props_ = ('shape', )\n\n        shape = 'circle'\n\n        group = 'window'\n\n        _config_props = ('group', 'size')\n\nwhen run with:\n\n.. code-block:: python\n\n    app = App()\n    # now get and save config to yaml file\n    dump_config('hook_properties.yaml', read_config_from_object(app))\n\nwill generate this yaml file:\n\n.. code-block:: yaml\n\n    app panel:\n      color: A4FF67\n      name: Desk\n      size: 12\n    home panel:\n      group: window\n      size: 12\n\nNotice how ``app panel`` contains the properties\nof both ``RootPanel`` and ``AppPanel``, while ``home panel`` only has the\nproperties listed in ``_config_props``. ``_config_children`` behaves\nsimilarly.\n\nCustom values hooks\n-------------------\n\nWe may wish to hook the property getting/setting process to\nchange the value before it is saved or before it is applied again.\n\nE.g. consider that we have a property that stores a namedtuple that we need\nto dump as a list (because yaml doesn't understand named tuple) and create\na named tuple again when restoring. ``get_config_property`` and\n``apply_config_property`` are the needed hook methods, that are\nautomatically used if present in the class:\n\n.. code-block:: python\n\n    from collections import namedtuple\n    from tree_config import dump_config, load_config, apply_config, \\\n        read_config_from_object\n\n    Point = namedtuple('Point', ['x', 'y'])\n\n\n    class App:\n\n        _config_props_ = ('point', 'name')\n\n        point = Point(11, 34)\n\n        name = ''\n\n        def get_config_property(self, name):\n            if name == 'point':\n                return tuple(self.point)\n            return getattr(self, name)\n\n        def apply_config_property(self, name, value):\n            if name == 'point':\n                self.point = Point(*value)\n            else:\n                setattr(self, name, value)\n\nThen, running:\n\n.. code-block:: python\n\n    from tree_config import dump_config, read_config_from_object\n    app = App()\n    dump_config('custom_value_example.yaml', read_config_from_object(app))\n    print(f'point is: {app.point}')\n\ncreates a ``custom_value_example.yaml`` file with the following contents:\n\n.. code-block:: yaml\n\n    name: ''\n    point: [11, 34]\n\nand it prints ``point is: Point(x=11, y=34)``. If we want to load and apply the\nyaml file again, we do:\n\n.. code-block:: python\n\n    from tree_config import load_config, apply_config\n    app = App()\n    apply_config(app, load_config(app, 'custom_value_example.yaml'))\n    print(f'point is: {app.point}')\n\nThis in turn prints again ``point is: Point(x=11, y=34)``.\n\nSee also ``apply_config_child`` for similarly hooking into applying the children\nobjects. The default, when not provided is to use ``apply_config``, so if\noverriding, that should probably also be used for the base case.\n\nCustom tags (pickling)\n^^^^^^^^^^^^^^^^^^^^^^\n\nYaml offers support for representing arbitrary objects using custom tags in the\nfile. This enables global support for the objects, without having to use\n``get_config_property`` / ``apply_config_property`` wherever they are used.\n\nUsing the point example above:\n\n.. code-block:: python\n\n    from collections import namedtuple\n    from tree_config import dump_config, load_config, apply_config, \\\n        read_config_from_object\n    from ruamel.yaml import BaseConstructor, BaseRepresenter\n\n    Point = namedtuple('Point', ['x', 'y'])\n\n    yaml_tag = '!tree_config_example_point'\n\n    # encoder\n    def _represent_point(representer: BaseRepresenter, val):\n        return representer.represent_sequence(yaml_tag, tuple(val))\n\n    # decoder\n    def _construct_point(constructor: BaseConstructor, tag, node):\n        return Point(*constructor.construct_sequence(node))\n\n    # tell yaml how to represent a Point\n    def register_point_yaml_support() -\u003e None:\n        BaseRepresenter.add_multi_representer(Point, _represent_point)\n        BaseConstructor.add_multi_constructor(yaml_tag, _construct_point)\n\n\n    class App:\n\n        _config_props_ = ('point', 'name')\n\n        point = Point(11, 34)\n\n        name = ''\n\nNow, call:\n\n.. code-block:: python\n\n    register_point_yaml_support()\n\nbefore running the tree-config dumping/loading code from the last section and\nit will generate a yaml file with contents:\n\n.. code-block:: yaml\n\n    name: ''\n    point: !tree_config_example_point [11, 34]\n\nSee also ``yaml_dumps`` and ``yaml_loads`` for additional customization.\nMost functions take a ``yaml_dump_str`` / ``yaml_load_str`` to allow further\ncustomizing the yaml objects. See also ``register_torch_yaml_support``\nin ``tree_config.yaml`` for a more complete example as well as some built-in\noptional representers that can be registered directly.\n\nPost-applying dispatch\n----------------------\n\nAfter applying configuration to a object and its children objects,\ntree-config will call the ``post_config_applied`` method of the object, if\nthe method exists. E.g.:\n\n.. code-block:: python\n\n    from tree_config import dump_config, load_config, apply_config, \\\n        read_config_from_object\n\n\n    class App:\n\n        _config_props_ = ('size', 'name')\n\n        _config_children_ = {'app panel': 'panel'}\n\n        size = 12\n\n        name = 'Desk'\n\n        def __init__(self):\n            self.panel = Panel()\n\n        def apply_config_property(self, name, value):\n            print('applying', name)\n            setattr(self, name, value)\n\n        def post_config_applied(self):\n            print('done applying app')\n\n\n    class Panel:\n\n        _config_props_ = ('color', )\n\n        color = 'A4FF67'\n\n        def apply_config_property(self, name, value):\n            print('applying', name)\n            setattr(self, name, value)\n\n        def post_config_applied(self):\n            print('done applying panel')\n\nThen, saving and again applying the yaml using:\n\n.. code-block:: python\n\n    # create app and set properties\n    app = App()\n\n    # now get and save config to yaml file\n    dump_config('post_apply_dispatch.yaml', read_config_from_object(app))\n    # load config and apply it\n    apply_config(app, load_config(app, 'post_apply_dispatch.yaml'))\n\nprints the following::\n\n    applying color\n    done applying panel\n    applying name\n    applying size\n    done applying app\n\nConfigurable class\n------------------\n\nThe above examples used a duck typing approach to these special configuration/hook\nmethods, and any/all of these methods were optional. tree-config also offers a\n``Configurable`` superclass that defines all these methods with appropriate\ndefault values.\n\nThere's no benefit to inheriting from ``Configurable``, but it does provide a\nbaseclass listing all the special configuration methods. Additionally,\nit does cache the list of properties/config children for each class,\nso once looked up, it does not need to walk the tree, unlike the duck\ntyping approach that re-computes at every save/apply.\n\nAuto docs\n=========\n\nIn addition to configuration, tree-config can also hook into the sphinx doc\ngenerating build steps and generate docs listing all the properties that\ncan be configured by the application and show the doc string for each of them.\nThis is helpful to users who want to configure these properties using the\nconfiguration yaml file.\n\nThe example directory has a complete doc example.\n\nGiven a root object (e.g. App in the examples), we can add callbacks in\n``conf.py`` that is called by sphinx as it encounters properties listed in\n``_config_props_``. The callback then saves the doc strings of these properties\ninto a yaml file.\n\nSubsequently, when the build is done, tree-config can go through all the\nconfigurable properties and starting from the root object or class, extract\nthe doc strings from the yaml file, and create a rst file of those docstrings.\n\nE.g. starting with this code in :\n\n.. code-block:: python\n\n    class App:\n        \"\"\"The app.\"\"\"\n\n        _config_props_ = ('size', 'name')\n\n        _config_children_ = {'app panel': 'panel1', 'home panel': 'panel2'}\n\n        size = 55\n        \"\"\"Some filename.\"\"\"\n\n        name = ''\n        \"\"\"Some name.\"\"\"\n\n        panel1: 'AppPanel' = None\n        \"\"\"The app panel.\"\"\"\n\n        panel2: 'HomePanel' = None\n        \"\"\"The home panel.\"\"\"\n\n        def __init__(self, size, name, color, shape):\n            self.size = size\n            self.name = name\n\n            self.panel1 = AppPanel()\n            self.panel1.color = color\n            self.panel2 = HomePanel()\n            self.panel2.shape = shape\n\n\n    class AppPanel:\n        \"\"\"The app panel.\"\"\"\n\n        _config_props_ = ('color', )\n\n        color = ''\n        \"\"\"Color of the app.\"\"\"\n\n\n    class HomePanel:\n        \"\"\"The home panel.\"\"\"\n\n        _config_props_ = ('shape', )\n\n        shape = ''\n        \"\"\"Shape of the home.\"\"\"\n\nthen, we add the following to the top of the ``conf.py`` file:\n\n.. code-block:: python\n\n    import os\n    import sys\n    from functools import partial\n    sys.path.insert(0, os.path.abspath('../'))\n    from config_example import App\n    from tree_config.doc_gen import create_doc_listener, write_config_props_rst\n\nthe exact path added to ``sys.path`` depends on where the code is, or if it's a python\npackage that is not needed because it's already installed.\n\nWe also need to add ``'sphinx.ext.autodoc'`` to the list of extensions. Finally,\nat the end of ``conf.py`` add:\n\n.. code-block:: python\n\n    def setup(app):\n        # dump all config_example package/subpackages config docstrings to config_prop_docs.yaml\n        create_doc_listener(app, 'config_example', 'config_prop_docs.yaml')\n\n        # then get docstrings from yaml file, walk all config properties from App and\n        # dump formatted config docs to source/config.rst\n        app.connect(\n            'build-finished', partial(\n                write_config_props_rst, App, 'config_example',\n                filename='config_prop_docs.yaml', rst_filename='source/config.rst')\n        )\n\nFinally, to the sphinx generated ``index.rst`` we added ``config.rst`` (the filename\nof the file that will be automatically created under source).\nWe also need to add somewhere in the index or files it references the auto-doc\nreferences for all the classes, otherwise we won't get the relevant docstrings.\nWe added it as:\n\n.. code-block:: rst\n\n    .. toctree::\n       :maxdepth: 2\n       :caption: Contents:\n\n       config.rst\n\n\n    API\n    ===\n\n    .. automodule:: config_example\n       :members:\n\nin ``index.rst``.\n\nFinally, we run:\n\n.. code-block:: shell\n\n    echo $'Config\\n===========' \u003e source/config.rst\n    make html\n    make html\n\nFirst we created a mostly empty config.rst file. Otherwise sphinx doesn't\ninclude it when it is generated. Next we ran ``make html`` twice, the first\ntime it automatically generates the following ``config_prop_docs.yaml`` file:\n\n.. code-block:: yaml\n\n    config_example.App:\n      name:\n      - Some name.\n      - ''\n      size:\n      - Some filename.\n      - ''\n    config_example.AppPanel:\n      color:\n      - Color of the app.\n      - ''\n    config_example.HomePanel:\n      shape:\n      - Shape of the home.\n      - ''\n\nThe second ``make html`` extracts the docstrings from this yaml file and\nuses that create ``config.rst`` with the following contents:\n\n.. code-block:: rst\n\n    CONFIG_EXAMPLE Config\n    =====================\n\n    The following are the configuration options provided by the app. It can be configured by changing appropriate values in the ``config.yaml`` settings file. The options default to the default value of the classes for each of the options.\n\n    `name`:\n     Default value::\n\n      ''\n\n     Some name.\n\n    `size`:\n     Default value::\n\n      55\n\n     Some filename.\n\n\n    home panel\n    ----------\n\n    `shape`:\n     Default value::\n\n      ''\n\n     Shape of the home.\n\n\n    app panel\n    ---------\n\n    `color`:\n     Default value::\n\n      ''\n\n     Color of the app.\n\nThis rst is automatically rendered by sphinx to nice html with the rest of the docs and\nit looks something like:\n\n----\n\nCONFIG_EXAMPLE Config\n=====================\n\nThe following are the configuration options provided by the app. It can be configured by changing appropriate values in the ``config.yaml`` settings file. The options default to the default value of the classes for each of the options.\n\n`name`:\n Default value::\n\n  ''\n\n Some name.\n\n`size`:\n Default value::\n\n  55\n\n Some filename.\n\n\nhome panel\n----------\n\n`shape`:\n Default value::\n\n  ''\n\n Shape of the home.\n\n\napp panel\n---------\n\n`color`:\n Default value::\n\n  ''\n\n Color of the app.\n\n----\n\nClass vs instance\n-----------------\n\nThe configuration examples above save the config from the App *instance*.\nOne can also use the App *class* to dump the yaml. The major difference is that the\n``apply_config_child``, ``get_config_property``, ``apply_config_property``,\nand ``post_config_applied`` methods, which are instance methods, are skipped and\nnot used.\n\nAlso, unlike for instances, where it would fail if ``_config_children_`` lists\na child property whose value is None, for the class it will fallback on the type\nhint of the property, if one is defined.\n\nUsing the ``App`` class, rather than a ``App()`` instance is helpful during doc\nbuilding when it may not be possible to instantiate the full App\n(see the docs example above that uses the class instance with type hints).\n\nReusing other project docs\n--------------------------\n\nBecause we rely on autodoc to generate ``config_prop_docs.yaml``, tree-config\nprovides a mechanism to reuse the docstrings from other projects we depend on.\n\nE.g. imagine we depend on ``remote1`` and ``remote2`` projects who defines classes\nthat is configurable and our projects inherits and extends them with further\nconfigurable properties.\nAlso assume these remote projects dumped their configurable docstrings to\n``config_prop_docs.yaml`` like in the example and made it available in the\nroot of their sphinx generated docs e.g. on github-pages.\n\nThen, tree-config provides tools to merge those docstrings into ours to be able\nto create ``config.rst`` from them as follows:\n\n.. code-block:: shell\n\n    echo $'Config\\n===========' \u003e source/config.rst\n    python -m tree_config.doc_gen download \\\n        -u \"https://user.github.io/remote1/config_prop_docs.yaml\" -o config_prop_docs.yaml\n    python -m tree_config.doc_gen download -f config_prop_docs.yaml \\\n        -u \"https://matham.github.io/remote2/config_prop_docs.yaml\" -o config_prop_docs.yaml\n    make html\n    make html\n\nThis downloads and merges the yaml files from our dependencies, adds to it our own\ndocs, and generates the ``config.rst``.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatham%2Ftree-config","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmatham%2Ftree-config","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmatham%2Ftree-config/lists"}