{"id":15951511,"url":"https://github.com/timvink/config_decorator","last_synced_at":"2026-02-16T00:30:18.506Z","repository":{"id":66131532,"uuid":"366366143","full_name":"timvink/config_decorator","owner":"timvink","description":"Demonstrate how to use a config in an ML package","archived":false,"fork":false,"pushed_at":"2021-05-12T12:07:54.000Z","size":7,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-12-31T00:27:29.441Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/timvink.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-05-11T11:59:41.000Z","updated_at":"2021-05-30T20:25:10.000Z","dependencies_parsed_at":null,"dependency_job_id":"f877b341-cb1b-4527-ab07-5283da876df6","html_url":"https://github.com/timvink/config_decorator","commit_stats":{"total_commits":3,"total_committers":1,"mean_commits":3.0,"dds":0.0,"last_synced_commit":"a714ff4aabb0a393ce329f48b9c2c6958d681d83"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvink%2Fconfig_decorator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvink%2Fconfig_decorator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvink%2Fconfig_decorator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/timvink%2Fconfig_decorator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/timvink","download_url":"https://codeload.github.com/timvink/config_decorator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239824983,"owners_count":19703199,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-07T13:02:24.872Z","updated_at":"2026-02-16T00:30:18.444Z","avatar_url":"https://github.com/timvink.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Configuration handling in machine learning projects\n\nThis repo is a demo on reading configuration files that are included in a python package and added to the scope of functions using decorators.\n\n## Problem\n\nBuild and maintain a couple of machine learning projects, and you'll realise some good configuration handling is crucial.\n\n- How to package the config along with your code\n- How to make the configuration settings available where you need them without being too verbose\n- How to support auto-reloading\n\nYou can solve these problems elegantly with `pkgutils` and a decorator.\n\n## Solution\n\nAdd your `conf` folder inside your python package. Ours will be called `demo_config_decorator`:\n\n```\n.\n├── MANIFEST.in\n├── README.md\n├── demo_config_decorator\n│   ├── __init__.py\n│   ├── conf                  \u003c----- Folder containing configuration.\n│   │   └── base.yaml\n│   ├── config.py\n│   └── demo.py\n└── setup.py\n```\n\nAdd a `MANIFEST.in` in the root of your repository like so:\n\n```\ndemo_config_decorator/conf/*.yaml\n```\n\nMake sure your package includes package data by updating `setup.py` with:\n\n```python\n# setup.py\nsetup(\n    # ...\n    include_package_data=True,\n    # ...\n)\n```\n\nThen you can read your configuration using [`pkgutil`](https://docs.python.org/3/library/pkgutil.html) anywhere inside your package:\n\n```python\n# config.py\nimport pkgutil\n\ndef read_config():\n    c = pkgutil.get_data(__name__, \"conf/base.yaml\")    \n    return yaml.safe_load(c)\n```\n\nNow in any python file inside your package, you can add:\n\n```python\nfrom demo_config_fsdfs.config import read_config\nSETTINGS = read_config()\n```\n\nThe problem however is that this does not [auto-reload](https://godatadriven.com/blog/write-less-terrible-code-with-jupyter-notebook/), which is useful when you are using jupyter notebooks. This means that if you update the config, you'll need to restart the kernel to reload the `SETTINGS` global. You can fix this by adding a `settings = read_config()` inside each of your functions that need it.\n\nHowever, now cannot change the `settings` depending on the context (for example a different set of `settings` when unit testing). You could choose to add a `settings` parameter to your function and add a [decorator](https://realpython.com/primer-on-python-decorators) to fill it:\n\n```python\n# config.py\nimport functools\n\ndef add_settings(func):\n    @functools.wraps(func)\n    def wrapper(*args, **kwargs):\n        if 'settings' not in kwargs:\n            kwargs['settings'] = read_config()\n        return func(*args, **kwargs)\n    return wrapper\n```\n\nWhich you can use like this:\n\n```python\n# demo.py\n@add_settings\ndef say_hello(name, *, settings):\n    return f\"Hello, {name}! This is a date from the settings: {settings.get('date1')}.\"\n\nsay_hello('World')\n```\n\nThis approach is nice because:\n\n1) The decorator will fill in the `settings` argument if the user does not supply it.\n1) The `*, settings` in `say_hello()` [enforces use of a keyword argument](https://stackoverflow.com/questions/2965271/forced-naming-of-parameters-in-python) for `settings`, preventing errors.\n1) You can explicitly pass a different set of `settings` to the function using a named argument, which is useful for unit testing or in production.\n1) The approach supports auto-reloading (because the `read_config()` file is called on every function call)\n1) Because the config is packaged along with your code, it is easy to distribute.\n\nTo get started with a working example, you can fork and clone this repository.\n\nIf this approach doesn't suit your needs, consider adding the parts of the config that you need inside a function as an explicit parameter. That's the approach [gin-config](https://github.com/google/gin-config) takes, a python package built by google explicitly for machine learning configuration management. There's also a great [intro to gin](https://calmcode.io/gin/intro-to-gin.html) video  =by [calmcode](https://calmcode.io/).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvink%2Fconfig_decorator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftimvink%2Fconfig_decorator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftimvink%2Fconfig_decorator/lists"}