Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lincolnloop/goodconf

Transparently load variables from environment or JSON/YAML/TOML file.
https://github.com/lincolnloop/goodconf

config configuration django env environment-variables json pydantic python toml yaml

Last synced: 5 days ago
JSON representation

Transparently load variables from environment or JSON/YAML/TOML file.

Awesome Lists containing this project

README 

        
Goodconf

========



.. image:: https://github.com/lincolnloop/goodconf/actions/workflows/test.yml/badge.svg?branch=main&event=push

    :target: https://github.com/lincolnloop/goodconf/actions/workflows/test.yml?query=branch%3Amain+event%3Apush



.. image:: https://results.pre-commit.ci/badge/github/lincolnloop/goodconf/main.svg

    :target: https://results.pre-commit.ci/latest/github/lincolnloop/goodconf/main

    :alt: pre-commit.ci status



.. image:: https://img.shields.io/codecov/c/github/lincolnloop/goodconf.svg

    :target: https://codecov.io/gh/lincolnloop/goodconf



.. image:: https://img.shields.io/pypi/v/goodconf.svg

    :target: https://pypi.python.org/pypi/goodconf



.. image:: https://img.shields.io/pypi/pyversions/goodconf.svg

    :target: https://pypi.python.org/pypi/goodconf



A thin wrapper over `Pydantic's settings management `__.

Allows you to define configuration variables and load them from environment or JSON/YAML/TOML

file. Also generates initial configuration files and documentation for your

defined configuration.



Installation

------------



``pip install goodconf`` or ``pip install goodconf[yaml]`` /

``pip install goodconf[toml]`` if parsing/generating YAML/TOML

files is required. When running on Python 3.11+ the ``[toml]``

extra is only required for generating TOML files as parsing

is supported natively.



Quick Start

-----------



Let's use configurable Django settings as an example.



First, create a ``conf.py`` file in your project's directory, next to

``settings.py``:



.. code:: python



    import base64

    import os



    from goodconf import GoodConf, Field

    from pydantic import PostgresDsn



    class AppConfig(GoodConf):

        "Configuration for My App"

        DEBUG: bool

        DATABASE_URL: PostgresDsn = "postgres://localhost:5432/mydb"

        SECRET_KEY: str = Field(

            initial=lambda: base64.b64encode(os.urandom(60)).decode(),

            description="Used for cryptographic signing. "

            "https://docs.djangoproject.com/en/2.0/ref/settings/#secret-key")



        model_config = {"default_files": ["/etc/myproject/myproject.yaml", "myproject.yaml"]}



    config = AppConfig()



Next, use the config in your ``settings.py`` file:



.. code:: python



    import dj_database_url

    from .conf import config



    config.load()



    DEBUG = config.DEBUG

    SECRET_KEY = config.SECRET_KEY

    DATABASES = {"default": dj_database_url.parse(config.DATABASE_URL)}



In your initial developer installation instructions, give some advice such as:



.. code:: shell



    python -c "import myproject; print(myproject.conf.config.generate_yaml(DEBUG=True))" > myproject.yaml



Better yet, make it a function and `entry point `__ so you can install

your project and run something like ``generate-config > myproject.yaml``.



Usage

-----



``GoodConf``

^^^^^^^^^^^^



Your subclassed ``GoodConf`` object can include a ``model_config`` dictionary with the following

attributes:



``file_env_var``

  The name of an environment variable which can be used for

  the name of the configuration file to load.

``default_files``

  If no file is passed to the ``load`` method, try to load a

  configuration from these files in order.



It also has one method:



``load``

  Trigger the load method during instantiation. Defaults to False.



Use plain-text docstring for use as a header when generating a configuration

file.



Environment variables always take precedence over variables in the configuration files.



See Pydantic's docs for examples of loading:



* `Dotenv (.env) files `_

* `Docker secrets `_



Fields

^^^^^^



Declare configuration values by subclassing ``GoodConf`` and defining class

attributes which are standard Python type definitions or Pydantic ``FieldInfo``

instances generated by the ``Field`` function.



Goodconf can use one extra argument provided to the ``Field`` to define an function

which can generate an initial value for the field:



``initial``

  Callable to use for initial value when generating a config



Django Usage

------------



A helper is provided which monkey-patches Django's management commands to

accept a ``--config`` argument. Replace your ``manage.py`` with the following:



.. code:: python



    # Define your GoodConf in `myproject/conf.py`

    from myproject.conf import config



    if __name__ == '__main__':

        config.django_manage()



Why?

----



I took inspiration from `logan `__ (used by

Sentry) and `derpconf `__ (used by

Thumbor). Both, however used Python files for configuration. I wanted a safer

format and one that was easier to serialize data into from a configuration

management system.



Environment Variables

^^^^^^^^^^^^^^^^^^^^^



I don't like working with environment variables. First, there are potential

security issues:



1. Accidental leaks via logging or error reporting services.

2. Child process inheritance (see `ImageTragick `__

   for an idea why this could be bad).



Second, in practice on deployment environments, environment variables end up

getting written to a number of files (cron, bash profile, service definitions,

web server config, etc.). Not only is it cumbersome, but also increases the

possibility of leaks via incorrect file permissions.



I prefer a single structured file which is explicitly read by the application.

I also want it to be easy to run my applications on services like Heroku

where environment variables are the preferred configuration method.



This module let's me do things the way I prefer in environments I control, but

still run them with environment variables on environments I don't control with

minimal fuss.



Contribute

----------



Create virtual environment and install package and dependencies.



.. code:: shell



    pip install -e ".[tests]"



Run tests



.. code:: shell



    pytest



Releases are done with GitHub Actions whenever a new tag is created. For more information,

see `<./.github/workflows/build.yml>`_