Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/MartinThoma/cfg_load
Load config files the right way
https://github.com/MartinThoma/cfg_load
configuration-files mit-license python3
Last synced: 3 months ago
JSON representation
Load config files the right way
- Host: GitHub
- URL: https://github.com/MartinThoma/cfg_load
- Owner: MartinThoma
- License: mit
- Created: 2018-02-24T20:50:32.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-03-15T21:01:17.000Z (over 1 year ago)
- Last Synced: 2024-07-14T15:04:43.731Z (4 months ago)
- Topics: configuration-files, mit-license, python3
- Language: Python
- Homepage:
- Size: 140 KB
- Stars: 41
- Watchers: 5
- Forks: 13
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://badge.fury.io/py/cfg-load)
[](https://pypi.org/project/cfg_load/)
[](http://cfg-load.readthedocs.io/en/latest/)
[](https://travis-ci.org/MartinThoma/cfg_load)
[](https://coveralls.io/github/MartinThoma/cfg_load?branch=master)
[](https://github.com/psf/black)
# cfg_load
Loading configuration files is a common task in many projects. This package
does the job.
## Installation
The recommended way to install cfg_load is:
```bash
$ pip install cfg_load[all] --user
```
Note: You might have to escape `[` and `]` in some shells like ZSH.
If you want the latest version:
```bash
$ git clone https://github.com/MartinThoma/cfg_load.git; cd cfg_load
$ pip install -e .[all] --user
```
## Usage
`cfg_load` is intended to be used as a library. In your code, it will mostly
be used like this:
```python
import cfg_load
config = cfg_load.load("some/path.yaml")
```
In order to check if it is doing what you expect, you can use it as a command
line tool:
```bash
$ cfg_load tests/examples/cifar10_baseline.yaml
{ 'dataset': { 'script_path': '/home/moose/GitHub/cfg_loader/datasets/cifar10_keras.py'},
'evaluate': { 'augmentation_factor': 32,
'batch_size': 1000,
'data_augmentation': { 'channel_shift_range': 0,
'featurewise_center': False,
'height_shift_range': 0.15,
'horizontal_flip': True,
'rotation_range': 0,
'samplewise_center': False,
'samplewise_std_normalization': False,
'shear_range': 0,
'vertical_flip': False,
'width_shift_range': 0.15,
'zca_whitening': False,
'zoom_range': 0}},
'model': { 'script_path': '/home/moose/GitHub/cfg_loader/models/baseline.py'},
'optimizer': { 'initial_lr': 0.0001,
'script_path': '/home/moose/GitHub/cfg_loader/optimizers/adam_keras.py'},
'train': { 'artifacts_path': '/home/moose/GitHub/cfg_loader/artifacts/cifar10_baseline',
'batch_size': 64,
'data_augmentation': { 'channel_shift_range': 0,
'featurewise_center': False,
'height_shift_range': 0.1,
'horizontal_flip': True,
'rotation_range': 0,
'samplewise_center': False,
'samplewise_std_normalization': False,
'shear_range': 0,
'vertical_flip': False,
'width_shift_range': 0.1,
'zca_whitening': False,
'zoom_range': 0},
'epochs': 1000,
'script_path': '/home/moose/GitHub/cfg_loader/train/train_keras.py'}}
```
You can see that it automatically detected that the file is a YAML file and
when you compare it to `cfg_load examples/cifar10_baseline.yaml --raw` you can
also see that it made the paths absolute.
## Good Application Practice
```python
import cfg_load
# Load defaults
base_cfg = cfg_load.load("some/path.yaml")
# Overwrite defaults if user defined it
user_cfg = cfg_load.load("other/path.yaml")
user_cfg = base_cfg.update(user_cfg)
# Overwrite user default with environment variables
env_mapping = cfg_load.load("other/env_mapping.yaml")
cfg = user_cfg.apply_env(env_mapping)
```
## Features
* You load your config like this: `cfg = cfg_load.load('examples/test.json')`
* No key that starts with `_` will ever be touched.
* Keys ending in `_path` will be made absolute.
* Don't worry about Unicode.
* Every key `[something]_module_path` triggers `cfg_load` to load the
file found at `[something]_module_path` as a Python module to
`cfg.modules['something']`.
* If an environment variable with the same name as a config key exists, the
take the value of the environment variable. *Please note*: If the type of
the overwritten key is not str, then `cfg_load` applies `json.loads` to the
environment variable.
* Every key ending with `_load_url` has to have `source_url` and `sink_path`.
Files from `source_url` will be loaded automatically and stored in the
`sink_path`. A `policy` parameter can specify if it should be `load_always`
or `load_if_missing`.
Not there, but planned fo the future:
* Every key `[something]_cfg_path` will trigger `cfg_load` to search for
another config file and append it at `[something]`. By this way you can
define configuration files recursively.
## Development
Check tests with `tox`.