{"id":16648684,"url":"https://github.com/makukha/pydantic-file-secrets","last_synced_at":"2025-04-09T16:50:46.392Z","repository":{"id":253300669,"uuid":"839851455","full_name":"makukha/pydantic-file-secrets","owner":"makukha","description":"Use file secrets in nested pydantic-settings models instead of built-in SecretsSettingsSource","archived":false,"fork":false,"pushed_at":"2025-03-07T22:15:35.000Z","size":183,"stargazers_count":8,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-23T18:54:19.507Z","etag":null,"topics":["config","configuration","docker","docker-secret","file-secrets","pydantic","pydantic-settings","pydantic-v2","python","python3","secrets","settings","validation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/makukha.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-08T13:00:52.000Z","updated_at":"2025-03-18T00:09:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"c8a93680-8853-46a7-a97d-ea285821ef6c","html_url":"https://github.com/makukha/pydantic-file-secrets","commit_stats":null,"previous_names":["makukha/pydantic-file-secrets"],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makukha%2Fpydantic-file-secrets","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makukha%2Fpydantic-file-secrets/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makukha%2Fpydantic-file-secrets/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/makukha%2Fpydantic-file-secrets/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/makukha","download_url":"https://codeload.github.com/makukha/pydantic-file-secrets/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248072615,"owners_count":21043266,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["config","configuration","docker","docker-secret","file-secrets","pydantic","pydantic-settings","pydantic-v2","python","python3","secrets","settings","validation"],"created_at":"2024-10-12T09:05:14.975Z","updated_at":"2025-04-09T16:50:46.386Z","avatar_url":"https://github.com/makukha.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pydantic-file-secrets 🔑\n[![Pydantic v2](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/pydantic/pydantic/main/docs/badge/v2.json)](https://pydantic.dev)\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: exec yq '\"\u003e \" + .project.description' pyproject.toml --\u003e\n\u003e Use file secrets in nested pydantic-settings models instead of built-in SecretsSettingsSource\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: include docs/badges.md --\u003e\n[![license](https://img.shields.io/github/license/makukha/pydantic-file-secrets.svg)](https://github.com/makukha/pydantic-file-secrets/blob/main/LICENSE)\n[![pypi](https://img.shields.io/pypi/v/pydantic-file-secrets.svg#v0.4.2)](https://pypi.org/project/pydantic-file-secrets)\n[![python versions](https://img.shields.io/pypi/pyversions/pydantic-file-secrets.svg)](https://pypi.org/project/pydantic-file-secrets)\n[![tests](https://raw.githubusercontent.com/makukha/pydantic-file-secrets/v0.4.2/docs/img/badge/tests.svg)](https://github.com/makukha/pydantic-file-secrets)\n[![coverage](https://raw.githubusercontent.com/makukha/pydantic-file-secrets/v0.4.2/docs/img/badge/coverage.svg)](https://github.com/makukha/pydantic-file-secrets)\n[![tested with multipython](https://img.shields.io/badge/tested_with-multipython-x)](https://github.com/makukha/multipython)\n[![uses docsub](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/makukha/docsub/refs/heads/main/docs/badge/v1.json)](https://github.com/makukha/docsub)\n[![mypy](https://img.shields.io/badge/type_checked-mypy-%231674b1)](http://mypy.readthedocs.io)\n[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/ruff)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![OpenSSF Best Practices](https://www.bestpractices.dev/projects/10108/badge)](https://www.bestpractices.dev/projects/10108)\n\u003c!-- docsub: end --\u003e\n[![pypi downloads](https://img.shields.io/pypi/dw/pydantic-file-secrets)](https://pypistats.org/packages/pydantic-file-secrets)\n\n\nThis project is inspired by discussions in Pydantic Settings repository and proposes solution to [#30](https://github.com/pydantic/pydantic-settings/issues/30) and [#154](https://github.com/pydantic/pydantic-settings/issues/154).\n\n\n# Features\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: include docs/features.md --\u003e\n* Plain or nested directory layout: `secrets/dir__key` or `secrets/dir/key`\n* Respects `env_prefix`, `env_nested_delimiter` and other [config options](https://github.com/makukha/pydantic-file-secrets?tab=readme-ov-file#configuration-options)\n* Implements config options `secrets_prefix`, `secrets_nested_delimiter` [and more](https://github.com/makukha/pydantic-file-secrets?tab=readme-ov-file#configuration-options) to configure secrets and env vars independently\n* Drop-in replacement of standard `SecretsSettingsSource`\n* Pure Python thin wrapper over standard `EnvSettingsSource`\n* No third party dependencies except `pydantic-settings`\n* Fully typed\n* 100% test coverage\n\u003c!-- docsub: end --\u003e\n\n\n# Installation\n\n```shell\n$ pip install pydantic-file-secrets\n```\n\n\n# Motivation\n\nNested Pydantic config can contain nested models with secret entries, as well as secrets in top level config. In dockerized environment, these entries may be read from file system, e.g. `/run/secrets` when using Docker Secrets:\n\n```python\nfrom pydantic import BaseModel, Secret\nfrom pydantic_settings import BaseSettings, SettingsConfigDict\n\nclass DbSettings(BaseModel):\n    user: str\n    passwd: Secret[str]  # secret in nested model\n\nclass Settings(BaseSettings):\n    app_key: Secret[str]  # secret in root model\n    db: DbSettings\n\n    model_config = SettingsConfigDict(\n        secrets_dir='/run/secrets',\n    )\n```\n\n\n# Usage\n\n\u003c!-- docsub: begin #usage.md --\u003e\n\u003c!-- docsub: include docs/usage.md --\u003e\n## Plain secrets directory layout\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x dirtree tests/test_usage.py:UsagePlain.secrets_dir --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```text\n📂 secrets\n├── 📄 app_key\n└── 📄 db__passwd\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: include tests/usage/plain.py --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```python\nfrom pydantic import BaseModel, SecretStr\nfrom pydantic_file_secrets import FileSecretsSettingsSource, SettingsConfigDict\nfrom pydantic_settings import BaseSettings\nfrom pydantic_settings.sources import PydanticBaseSettingsSource\n\n\nclass DbSettings(BaseModel):\n    passwd: SecretStr\n\n\nclass Settings(BaseSettings):\n    app_key: SecretStr\n    db: DbSettings\n\n    model_config = SettingsConfigDict(\n        secrets_dir='secrets',\n        secrets_nested_delimiter='__',\n    )\n\n    @classmethod\n    def settings_customise_sources(\n        cls,\n        settings_cls: type[BaseSettings],\n        init_settings: PydanticBaseSettingsSource,\n        env_settings: PydanticBaseSettingsSource,\n        dotenv_settings: PydanticBaseSettingsSource,\n        file_secret_settings: PydanticBaseSettingsSource,\n    ) -\u003e tuple[PydanticBaseSettingsSource, ...]:\n        return (\n            init_settings,\n            env_settings,\n            dotenv_settings,\n            FileSecretsSettingsSource(file_secret_settings),\n        )\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x cases tests/test_usage.py UsagePlain --\u003e\n```pycon\n\u003e\u003e\u003e Settings().model_dump()\n{'app_key': SecretStr('**********'), 'db': {'passwd': SecretStr('**********')}}\n```\n\n\u003c!-- docsub: end --\u003e\n\n\n## Nested secrets directory layout\n\nConfig option `secrets_nested_delimiter` overrides `env_nested_delimiter` for files. In particular, this allows to use nested directory layout along with environmemt variables for other non-secret settings:\n\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x dirtree tests/test_usage.py:UsageNested.secrets_dir --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```text\n📂 secrets\n├── 📄 app_key\n└── 📂 db\n    └── 📄 passwd\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: exec sed -n '/ *model_config =/,/ *)/p' tests/usage/nested.py --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```python\n    model_config = SettingsConfigDict(\n        secrets_dir='secrets',\n        secrets_nested_subdir=True,\n    )\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x cases tests/test_usage.py UsageNested --\u003e\n```pycon\n\u003e\u003e\u003e Settings().model_dump()\n{'app_key': SecretStr('**********'), 'db': {'passwd': SecretStr('**********')}}\n```\n\n\u003c!-- docsub: end --\u003e\n\n\n## Multiple `secrets_dir`\n\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x dirtree tests/test_usage.py:UsageMultiple.secrets_dir --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```text\n📂 secrets\n├── 📂 layer1\n│   └── 📄 app_key\n└── 📂 layer2\n    └── 📄 db__passwd\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: exec sed -n '/ *model_config =/,/ *)/p' tests/usage/multiple.py --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```python\n    model_config = SettingsConfigDict(\n        secrets_dir=['secrets/layer1', 'secrets/layer2'],\n        secrets_nested_delimiter='__',\n    )\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x cases tests/test_usage.py UsageMultiple --\u003e\n```pycon\n\u003e\u003e\u003e Settings().model_dump()\n{'app_key': SecretStr('**********'), 'db': {'passwd': SecretStr('**********')}}\n```\n\n\u003c!-- docsub: end --\u003e\n\n\n## Experimental syntactic sugar 🧪\n\n\u003e [!CAUTION]\n\u003e This syntax may change at any time. Pin current `pydantic-file-secrets` version if decided to use it.\n\nFew important things to note:\n\n- `@with_builtin_sources` decorator enables `NamedTuple` argument `src: BuiltinSources` encapsulating default builtins settings sources\n- `BaseSource` alias is shorter than `PydanticBaseSettingsSource` and is easier to use in type hints\n- `settings_cls` was removed from `settings_customise_sources` signature: `cls` seems to be sufficient\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: include tests/usage/sugar.py --\u003e\n\u003c!-- docsub: lines after 1 upto -1 --\u003e\n```python\nfrom pydantic import BaseModel, SecretStr\nfrom pydantic_file_secrets import (\n    BaseSource,\n    BuiltinSources,\n    FileSecretsSettingsSource,\n    SettingsConfigDict,\n    with_builtin_sources,\n)\nfrom pydantic_settings import BaseSettings\n\n\nclass DbSettings(BaseModel):\n    passwd: SecretStr\n\n\nclass Settings(BaseSettings):\n    app_key: SecretStr\n    db: DbSettings\n\n    model_config = SettingsConfigDict(\n        secrets_dir='secrets',\n        secrets_nested_delimiter='__',\n    )\n\n    @classmethod\n    @with_builtin_sources\n    def settings_customise_sources(cls, src: BuiltinSources) -\u003e tuple[BaseSource, ...]:\n        return (\n            src.init_settings,\n            src.env_settings,\n            src.dotenv_settings,\n            FileSecretsSettingsSource(src.file_secret_settings),\n        )\n```\n\u003c!-- docsub: end --\u003e\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x cases tests/test_usage.py UsageSugar --\u003e\n```pycon\n\u003e\u003e\u003e Settings().model_dump()\n{'app_key': SecretStr('**********'), 'db': {'passwd': SecretStr('**********')}}\n```\n\n\u003c!-- docsub: end --\u003e\n\u003c!-- docsub: end #usage.md --\u003e\n\n\n# Configuration options\n\n## secrets_dir\n\nPath to secrets directory. Same as `SecretsSettingsSource.secrets_dir` if `str` or `Path`. If `list`, the last match wins. If `secrets_dir` is passed in both source constructor and model config, values are not merged (constructor takes priority).\n\n## secrets_dir_missing\n\nIf `secrets_dir` does not exist, original `SecretsSettingsSource` issues a warning. However, this may be undesirable, for example if we don't mount Docker Secrets in e.g. dev environment. Now you have a choice:\n\n* `'ok'` — do nothing if `secrets_dir` does not exist\n* `'warn'` (default) — print warning, same as `SecretsSettingsSource`\n* `'error'` — raise `SettingsError`\n\nIf multiple `secrets_dir` passed, the same `secrets_dir_missing` action applies to each of them.\n\n## secrets_dir_max_size\n\nLimit the size of `secrets_dir` for security reasons, defaults to `SECRETS_DIR_MAX_SIZE` equal to 16 MiB.\n\n`FileSecretsSettingsSource` is a thin wrapper around [`EnvSettingsSource`](https://docs.pydantic.dev/latest/api/pydantic_settings/#pydantic_settings.EnvSettingsSource), which loads all potential secrets on initialization. This could lead to `MemoryError` if we mount a large file under `secrets_dir`.\n\nIf multiple `secrets_dir` passed, the limit applies to each directory independently.\n\n## secrets_case_sensitive\n\nSame as [`case_sensitive`](https://docs.pydantic.dev/latest/concepts/pydantic_settings/#case-sensitivity), but works for secrets only. If not specified, defaults to `case_sensitive`.\n\n## secrets_nested_delimiter\n\nSame as [`env_nested_delimiter`](https://docs.pydantic.dev/latest/concepts/pydantic_settings/#parsing-environment-variable-values), but works for secrets only. If not specified, defaults to `env_nested_delimiter`. This option is used to implement [nested secrets directory layout](https://github.com/makukha/pydantic-file-secrets?tab=readme-ov-file#nested-secrets-directory-layout) and allows to do even nastier things like `/run/secrets/model/delim/nested1/delim/nested2`.\n\n## secrets_nested_subdir\n\nBoolean flag to turn on nested secrets directory mode, `False` by default. If `True`, sets `secrets_nested_delimiter` to [`os.sep`](https://docs.python.org/3/library/os.html#os.sep). Raises `SettingsError` if `secrets_nested_delimiter` is already specified.\n\n## secrets_prefix\n\nSecret path prefix, similar to `env_prefix`, but works for secrets only. Defaults to `env_prefix` if not specified. Works in both plain and nested directory modes, like `'/run/secrets/prefix_model__nested'` and `'/run/secrets/prefix_model/nested'`.\n\n\n## Not supported config options\n\nSome config options that are declared in [`SecretsSettingsSource`](https://docs.pydantic.dev/latest/api/pydantic_settings/#pydantic_settings.SecretsSettingsSource) interface are actually [not working](https://github.com/makukha/pydantic-file-secrets/blob/main/tests/test_original_source.py) and are not supported in `FileSecretsSettingsSource`:\n\n* `env_ignore_empty`\n* `env_parse_none_str`\n* `env_parse_enums`\n\nHowever, we [make sure](https://github.com/makukha/pydantic-file-secrets/blob/main/tests/test_ignored_options.py) that the behaviour of `FileSecretsSettingsSource` matches `SecretsSettingsSource` to provide a drop-in replacement, although it is somewhat wierd (e.g. `env_parse_enums` is always `True`).\n\n\n# Testing\n\n100% test coverage is provided for latest Python and pydantic-settings version. Tests are run for all minor pydantic-settings v2 versions and all minor Python 3 versions supported by them:\n\n* `pyXY` — Python 3.{8,9,10,11,12,13}\n* `psXY` — pydantic-settings v2.{2,3,4,5,6,7,8}\n\n\u003c!-- docsub: begin --\u003e\n\u003c!-- docsub: x testres -f'py3{8..13};ps2{2..8}' -i'{\"fail\":\"❌\",\"pytest\":\"☑️\",\"mypy\":\"✅\",\"pytest-cov\":\"✳️\"}' .tox --\u003e\n|       |  ps28  |  ps27  |  ps26  |  ps25  |  ps24  |  ps23  |  ps22  |\n|-------|--------|--------|--------|--------|--------|--------|--------|\n| py313 |   ✳️   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n| py312 |   ✅   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n| py311 |   ✅   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n| py310 |   ✅   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n| py39  |   ✅   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n| py38  |   ✅   |   ✅   |   ✅   |   ✅   |   ☑️   |   ☑️   |   ☑️   |\n\u003c!-- docsub: end --\u003e\n\n- ✳️ pytest and mypy passing, coverage report generated\n- ✅ pytest and mypy passing\n- ☑️ pytest passing, mypy not attempted\n- ❌ tests failing or not attempted\n\n\n# History\n\n* September 2024 — [Multiple secrets_dir](https://github.com/pydantic/pydantic-settings/pull/372) feature was merged to [pydantic-settings v2.5.0](https://github.com/pydantic/pydantic-settings/releases/tag/v2.5.0)\n\n\n# Contributing\n\nPull requests, feature requests, and bug reports are welcome!\n\n* [Contribution guidelines](https://github.com/makukha/pydantic-file-secrets/blob/main/.github/CONTRIBUTING.md)\n\n\n# Authors\n\n* Michael Makukha\n\n\n# See also\n\n* [Documentation](https://github.com/makukha/pydantic-file-secrets#readme)\n* [Issues](https://github.com/makukha/pydantic-file-secrets/issues)\n* [Changelog](https://github.com/makukha/pydantic-file-secrets/blob/main/CHANGELOG.md)\n* [Security Policy](https://github.com/makukha/pydantic-file-secrets/blob/main/.github/SECURITY.md)\n* [Contribution Guidelines](https://github.com/makukha/pydantic-file-secrets/blob/main/.github/CONTRIBUTING.md)\n* [Code of Conduct](https://github.com/makukha/pydantic-file-secrets/blob/main/.github/CODE_OF_CONDUCT.md)\n* [License](https://github.com/makukha/pydantic-file-secrets/blob/main/LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakukha%2Fpydantic-file-secrets","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmakukha%2Fpydantic-file-secrets","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmakukha%2Fpydantic-file-secrets/lists"}