{"id":28544867,"url":"https://github.com/neoteroi/essentials-configuration","last_synced_at":"2025-07-30T09:03:22.292Z","repository":{"id":47916717,"uuid":"394355127","full_name":"Neoteroi/essentials-configuration","owner":"Neoteroi","description":"Implementation of key-value pair based configuration for Python applications.","archived":false,"fork":false,"pushed_at":"2023-12-28T10:54:35.000Z","size":55,"stargazers_count":20,"open_issues_count":0,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-07-06T21:41:30.630Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Neoteroi.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2021-08-09T16:00:52.000Z","updated_at":"2025-05-06T19:36:41.000Z","dependencies_parsed_at":"2025-07-06T21:34:26.307Z","dependency_job_id":"8380d544-bc71-4712-87c8-1eba08d17830","html_url":"https://github.com/Neoteroi/essentials-configuration","commit_stats":{"total_commits":8,"total_committers":1,"mean_commits":8.0,"dds":0.0,"last_synced_commit":"4ada96f8de7a76d85e2eff26bcbadf4427db67ae"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/Neoteroi/essentials-configuration","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neoteroi%2Fessentials-configuration","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neoteroi%2Fessentials-configuration/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neoteroi%2Fessentials-configuration/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neoteroi%2Fessentials-configuration/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Neoteroi","download_url":"https://codeload.github.com/Neoteroi/essentials-configuration/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Neoteroi%2Fessentials-configuration/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267842972,"owners_count":24153133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T22:38:03.547Z","updated_at":"2025-07-30T09:03:22.244Z","avatar_url":"https://github.com/Neoteroi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Build](https://github.com/Neoteroi/essentials-configuration/workflows/Build/badge.svg)\n[![pypi](https://img.shields.io/pypi/v/essentials-configuration.svg)](https://pypi.python.org/pypi/essentials-configuration)\n[![versions](https://img.shields.io/pypi/pyversions/essentials-configuration.svg)](https://github.com/Neoteroi/essentials-configuration)\n[![codecov](https://codecov.io/gh/Neoteroi/essentials-configuration/branch/main/graph/badge.svg?token=VzAnusWIZt)](https://codecov.io/gh/Neoteroi/essentials-configuration)\n[![license](https://img.shields.io/github/license/Neoteroi/essentials-configuration.svg)](https://github.com/Neoteroi/essentials-configuration/blob/main/LICENSE)\n\n# Python configuration utilities\nImplementation of key-value pair based configuration for Python applications.\n\n**Features:**\n- support for most common sources of application settings\n- support for overriding settings in sequence\n- support for nested structures and lists, using attribute notation\n- strategy to use environment specific settings\n- features to handle secrets and values stored in the user folder, for local\n  development\n- features to support validation of configuration items, for example using\n  `pydantic`, or user defined classes\n\nThis library is freely inspired by .NET Core `Microsoft.Extensions.Configuration` (_ref. [MSDN documentation](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-2.1), [Microsoft Extensions Configuration Deep Dive](https://www.paraesthesia.com/archive/2018/06/20/microsoft-extensions-configuration-deep-dive/)_).\n\nThe main class is influenced by Luciano Ramalho`s example of\nJSON structure explorer using attribute notation, in his book [Fluent Python](http://shop.oreilly.com/product/0636920032519.do).\n\n## Overview\n\n`essentials-configuration` provides a way to handle configuration roots\ncomposed of several layers, such as configuration files and environment\nvariables. Layers are applied in order and can override each others' values,\nenabling different scenarios like configuration by environment and system\ninstance.\n\n## Supported sources:\n- **toml** files\n- **yaml** files\n- **json** files\n- **ini** files\n- environment variables\n- secrets stored in the user folder, for development purpose\n- dictionaries\n- keys and values\n- [Azure Key Vault](https://docs.microsoft.com/en-us/azure/key-vault/general/basic-concepts), using [essentials-configuration-keyvault](https://github.com/Neoteroi/essentials-configuration-keyvault)\n- custom sources, implementing the `ConfigurationSource` interface\n\n## Installation\n\n```bash\npip install essentials-configuration\n```\n\nTo install with support for `YAML` configuration files:\n\n```\npip install essentials-configuration[yaml]\n```\n\nTo install with support for `YAML` configuration files and the `CLI` to handle\nuser secrets:\n\n```\npip install essentials-configuration[full]\n```\n\n## Extensions\n\n* Azure Key Vault secrets configuration source:\n  [essentials-configuration-keyvault](https://github.com/Neoteroi/essentials-configuration-keyvault)\n\n# Examples\n\nPlease read the list of examples in the [examples folder](./examples). Below\nare reported some of the examples that are tested in this repository.\n\n### TOML file\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.env import EnvVars\nfrom config.toml import TOMLFile\n\n\nbuilder = ConfigurationBuilder(\n    TOMLFile(\"settings.toml\"),\n    EnvVars(prefix=\"APP_\")\n)\n\nconfig = builder.build()\n```\n\nFor example, if the TOML file contains the following contents:\n\n```toml\ntitle = \"TOML Example\"\n\n[owner]\nname = \"Tom Preston-Werner\"\n```\n\nAnd the environment has a variable such as `APP_OWNER__NAME=AAA`, the owner\nname from the TOML file gets overridden by the env variable:\n\n```python\n\u003e\u003e\u003e config\n\u003cConfiguration {'title': '...', 'owner': '...'}\u003e\n\u003e\u003e\u003e config.title\n'TOML Example'\n\u003e\u003e\u003e config.owner.name\n'AAA'\n```\n\n### JSON file and environment variables\n\nIn the following example, configuration values will include the structure\ninside the file `settings.json` and environment variables whose name starts\nwith \"APP_\". Settings are applied in order, so environment variables with\nmatching name override values from the `json` file.\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.json import JSONFile\nfrom config.env import EnvVars\n\nbuilder = ConfigurationBuilder(\n    JSONFile(\"settings.json\"),\n    EnvVars(prefix=\"APP_\")\n)\n\nconfig = builder.build()\n```\n\nFor example, if the JSON file contains the following contents:\n\n```json\n{\n    \"logging\": {\n        \"level\": \"INFO\"\n    },\n    \"example\": \"Hello World\",\n    \"foo\": \"foo\"\n}\n```\n\nAnd the environment has a variable named `APP_foo=AAA`:\n\n```python\n\u003e\u003e\u003e config\n\u003cConfiguration {'logging': '...', 'example': '...', 'foo': '...'}\u003e\n\u003e\u003e\u003e config.foo\n'AAA'\n\u003e\u003e\u003e config.logging.level\n'INFO'\n```\n\n### YAML file and environment variables\n\nIn this example, configuration will include anything inside a file\n`settings.yaml` and environment variables. Settings are applied in order, so\nenvironment variables with matching name override values from the `yaml` file\n(using the `yaml` source requires also `PyYAML` package).\n\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.env import EnvVars\nfrom config.yaml import YAMLFile\n\nbuilder = ConfigurationBuilder()\n\nbuilder.add_source(YAMLFile(\"settings.yaml\"))\nbuilder.add_source(EnvVars())\n\nconfig = builder.build()\n```\n\n### YAML file, optional file by environment\n\nIn this example, if an environment variable with name `APP_ENVIRONMENT` and\nvalue `dev` exists, and a configuration file with name `settings.dev.yaml` is\npresent, it is read to override values configured in `settings.yaml` file.\n\n```python\nimport os\n\nfrom config.common import ConfigurationBuilder\nfrom config.env import EnvVars\nfrom config.yaml import YAMLFile\n\nenvironment_name = os.environ[\"APP_ENVIRONMENT\"]\n\nbuilder = ConfigurationBuilder(\n    YAMLFile(\"settings.yaml\"),\n    YAMLFile(f\"settings.{environment_name}.yaml\", optional=True)\n)\n\nconfig = builder.build()\n```\n\n### Filtering environment variables by prefix\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.env import EnvVars\n\nbuilder = ConfigurationBuilder()\n\nbuilder.add_source(EnvVars(prefix=\"APP_\"))\n\nconfig = builder.build()\n```\n\n### INI files\n\nINI files are parsed using the built-in `configparser` module, therefore\nsupport `[DEFAULT]` section; all values are kept as strings.\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.ini import INIFile\n\nbuilder = ConfigurationBuilder()\n\nbuilder.add_source(INIFile(\"settings.ini\"))\n\nconfig = builder.build()\n```\n\n### Dictionaries\n\n```python\nfrom config.common import ConfigurationBuilder\n\nbuilder = ConfigurationBuilder()\n\nbuilder.add_map({\"host\": \"localhost\", \"port\": 8080})\n\nbuilder.add_map({\"hello\": \"world\", \"example\": [{\"id\": 1}, {\"id\": 2}]})\n\nconfig = builder.build()\n\nassert config.host == \"localhost\"\nassert config.port == 8080\nassert config.hello == \"world\"\nassert config.example[0].id == 1\nassert config.example[1].id == 2\n```\n\n### Keys and values\n\n```python\nfrom config.common import ConfigurationBuilder\n\nbuilder = ConfigurationBuilder()\n\nbuilder.add_map({\"host\": \"localhost\", \"port\": 8080})\n\nbuilder.add_value(\"port\", 44555)\n\nconfig = builder.build()\n\nassert config.host == \"localhost\"\nassert config.port == 44555\n```\n\n### User secrets\n\nThe library provides a strategy to handle secrets during local development,\nstoring them into the user folder.\n\nThe following example shows how secrets can be configured for a project:\n\n```bash\nconfig settings init\nconfig settings set \"Foo\" \"Some secret value\"\n```\n\nSecrets are organized by project, and the project information is obtained from\n`pyproject.toml` files (from the `project.name` property). If `pyproject.toml`\nfile does not exist, one is generated automatically with a random name.\n\n---\n\nThen, from a Python app, it's possible to load the secrets from the user folder:\n\n```python\nfrom config.common import ConfigurationBuilder\nfrom config.json import JSONFile\nfrom config.secrets import UserSecrets\n\nbuilder = ConfigurationBuilder(JSONFile(\"settings.json\"), UserSecrets())\n\nconfig = builder.build()\n\nprint(config)\n# config contains both values from `settings.json`, and secrets read from the user\n# folder\n```\n\nSecrets are optional and should be used only for local development, they are\nstored in unencrypted form in the user's folder.\n\nProduction apps should use dedicated services to handle secrets, like\n[Azure Key Vault](https://docs.microsoft.com/en-us/azure/key-vault/general/basic-concepts),\n[AWS Secrets Manager](https://aws.amazon.com/secrets-manager/), or similar services.\nFor Azure Key Vault, an implementation is provided in [essentials-configuration-keyvault](https://github.com/Neoteroi/essentials-configuration-keyvault).\n\n## Handling user settings\n\nUser settings (stored in the user's folder) can be handled using the provided `config` CLI.\n\n![Rich CLI](https://gist.githubusercontent.com/RobertoPrevato/38a0598b515a2f7257c614938843b99b/raw/a83facd6eb4ddc1dc8552a5f5f073278470010c2/config-settings-rich-cli.png)\n\nThese settings can be useful to store secrets and other values during local development, or in general when working with desktop applications.\n\n### Overriding nested values\n\nIt is possible to override nested values by environment variables or\ndictionary keys using the following notation for sub properties:\n\n* keys separated by colon \":\", such as `a:d:e`\n* keys separated by \"__\", such as `a__d__e`\n\n```python\nfrom config.common import ConfigurationBuilder, MapSource\n\n\nbuilder = ConfigurationBuilder(\n    MapSource(\n        {\n            \"a\": {\n                \"b\": 1,\n                \"c\": 2,\n                \"d\": {\n                    \"e\": 3,\n                    \"f\": 4,\n                },\n            }\n        }\n    )\n)\n\nconfig = builder.build()\n\nassert config.a.b == 1\nassert config.a.d.e == 3\nassert config.a.d.f == 4\n\nbuilder.add_value(\"a:d:e\", 5)\n\nconfig = builder.build()\n\nassert config.a.d.e == 5\nassert config.a.d.f == 4\n\n```\n\n### Overriding nested values using env variables\n\n```python\nimport os\n\nfrom config.common import ConfigurationBuilder, MapSource\nfrom config.env import EnvVars\n\nbuilder = ConfigurationBuilder(\n    MapSource(\n        {\n            \"a\": {\n                \"b\": 1,\n                \"c\": 2,\n                \"d\": {\n                    \"e\": 3,\n                    \"f\": 4,\n                },\n            }\n        }\n    )\n)\n\nconfig = builder.build()\n\nassert config.a.b == 1\nassert config.a.d.e == 3\nassert config.a.d.f == 4\n\n# NB: if an env variable such as:\n# a:d:e=5\n# or...\n# a__d__e=5\n#\n# is defined, it overrides the value  from the dictionary\n\nos.environ[\"a__d__e\"] = \"5\"\n\nbuilder.sources.append(EnvVars())\n\nconfig = builder.build()\n\nassert config.a.d.e == \"5\"\n```\n\n### Overriding values in list items using env variables\n\n```python\nimport os\n\nfrom config.common import ConfigurationBuilder, MapSource\nfrom config.env import EnvVars\n\nbuilder = ConfigurationBuilder(\n    MapSource(\n        {\n            \"b2c\": [\n                {\"tenant\": \"1\"},\n                {\"tenant\": \"2\"},\n                {\"tenant\": \"3\"},\n            ]\n        }\n    ),\n    EnvVars(),\n)\n\nos.environ[\"b2c__0__tenant\"] = \"5\"\n\nconfig = builder.build()\n\nassert config.b2c[0].tenant == \"5\"\nassert config.b2c[1].tenant == \"2\"\nassert config.b2c[2].tenant == \"3\"\n```\n\n### Typed config\n\nTo bind configuration sections with types checking, for example to use `pydantic` to\nvalidate application settings, use the `config.bind` method like in\nthe following example:\n\n```yaml\n# example-01.yaml\nfoo:\n  value: \"foo\"\n  x: 100\n```\n\n```python\n# example\nfrom pydantic import BaseModel\n\nfrom config.common import ConfigurationBuilder\nfrom config.yaml import YAMLFile\n\n\nclass FooSettings(BaseModel):\n    value: str\n    x: int\n\n\nbuilder = ConfigurationBuilder(YAMLFile(\"example-01.yaml\"))\n\nconfig = builder.build()\n\n# the bind method accepts a variable number of fragments to\n# obtain the configuration section that should be used to instantiate the given type\nfoo_settings = config.bind(FooSettings, \"foo\")\n\nassert isinstance(foo_settings, FooSettings)\nassert foo_settings.value == \"foo\"\nassert foo_settings.x == 100\n```\n\n### Goal and non-goals\n\nThe goal of this package is to provide a way to handle configuration roots,\nfetching and composing settings from different sources, usually happening\nonce at application's start.\n\nThe library implements only a synchronous API and fetching of application\nsettings atomically (it doesn't support generators), like application settings\nfetched from INI, JSON, or YAML files that are read once in memory entirely.\nAn asynchronous API is currently out of the scope of this library, since its\nprimary use case is to fetch configuration values once at application's start.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneoteroi%2Fessentials-configuration","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fneoteroi%2Fessentials-configuration","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fneoteroi%2Fessentials-configuration/lists"}