{"id":13578611,"url":"https://github.com/PetrochukM/HParams","last_synced_at":"2025-04-05T19:33:19.575Z","repository":{"id":35390739,"uuid":"210755063","full_name":"PetrochukM/HParams","owner":"PetrochukM","description":"Configure Python functions explicitly and safely ","archived":false,"fork":false,"pushed_at":"2024-11-18T18:40:15.000Z","size":215,"stargazers_count":126,"open_issues_count":5,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-20T20:10:52.154Z","etag":null,"topics":["configuration","deep-learning","hyperparameters","machine-learning","python","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PetrochukM.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-09-25T04:25:51.000Z","updated_at":"2024-11-18T18:40:19.000Z","dependencies_parsed_at":"2024-01-23T11:01:27.330Z","dependency_job_id":"865cda49-6a18-44ce-b13d-0afea4da515a","html_url":"https://github.com/PetrochukM/HParams","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PetrochukM%2FHParams","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PetrochukM%2FHParams/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PetrochukM%2FHParams/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PetrochukM%2FHParams/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PetrochukM","download_url":"https://codeload.github.com/PetrochukM/HParams/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393094,"owners_count":20931804,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["configuration","deep-learning","hyperparameters","machine-learning","python","python3"],"created_at":"2024-08-01T15:01:32.223Z","updated_at":"2025-04-05T19:33:19.185Z","avatar_url":"https://github.com/PetrochukM.png","language":"Python","funding_links":[],"categories":["Python","Tooling"],"sub_categories":[],"readme":"# Config\n\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/pythonic-config.svg?style=flat-square)\n[![Codecov](https://img.shields.io/codecov/c/github/PetrochukM/HParams/master.svg?style=flat-square)](https://codecov.io/gh/PetrochukM/HParams)\n[![Downloads](http://pepy.tech/badge/pythonic-config)](http://pepy.tech/project/pythonic-config)\n[![Build Status](https://img.shields.io/travis/PetrochukM/HParams/master.svg?style=flat-square)](https://travis-ci.com/PetrochukM/HParams)\n[![License: MIT](https://img.shields.io/badge/License-MIT-brightgreen.svg?style=flat-square)](https://opensource.org/licenses/MIT)\n[![Twitter: PetrochukM](https://img.shields.io/twitter/follow/MPetrochuk.svg?style=social)](https://twitter.com/MPetrochuk)\n\nThis package allows you to configure functions explicitly and safely. You will be able to create an\nintuitive type-checked configuration file that directly sets function arguments, globally.\n\nThis is a lightweight package with only two widely used dependencies and only a couple hundred line\nof code.\n\n#### Contents\n\n- [Config](#config)\n      - [Contents](#contents)\n  - [Install](#install)\n  - [Usage 🤗](#usage-)\n    - [Configuring a function](#configuring-a-function)\n    - [Writing a configuration file](#writing-a-configuration-file)\n    - [Configuring via the command line](#configuring-via-the-command-line)\n    - [Logging the configuration](#logging-the-configuration)\n    - [Advanced: Sharing configurations between processes](#advanced-sharing-configurations-between-processes)\n    - [Advanced: Ensuring the configuration is used](#advanced-ensuring-the-configuration-is-used)\n    - [Advanced: Find unused configurations](#advanced-find-unused-configurations)\n\n## Install\n\nMake sure you have Python 3, then you can install `pythonic-config` using `pip`:\n\n```bash\npip install pythonic-config\n```\n\nInstall the latest code via:\n\n```bash\npip install git+https://github.com/PetrochukM/Config.git\n```\n\n## Usage 🤗\n\n### Configuring a function\n\nAny function can be configured, and then used anywhere, see below:\n\n```python\nimport config as cf\n\n# Define function\ndef do_something_cool(how_many_times: int):\n    pass\n\n# Configure function\ncf.add({do_something_cool: cf.Args(how_many_times=5)})\n\n# Use the configured function anywhere! 🎉\ndo_something_cool(how_many_times=cf.get())\n```\n\nThis approach is simple but powerful. Now, each configuration can be directly attributed to a\ndocumented function argument.\n\nFurthermore, `config` incorporates `typeguard` 💂‍♀️ so every configuration is type checked at runtime.\n\n### Writing a configuration file\n\nThe simple example above can be extended to create a configuration file, for example:\n\n```python\nimport config as cf\nimport data\nimport train\n\ncf.add({\n  data.get_data: cf.Args(\n      train_data_path=\"url_lists/all_train.txt\",\n      val_data_path=\"url_lists/all_val.txt\"\n  ),\n  data.dataset_reader: cf.Args(\n      type_=\"cnn_dm\",\n      source_max_tokens=1022,\n      target_max_tokens=54,\n  ),\n  train.make_model: cf.Args(type_=\"bart\"),\n  train.Trainer.make_optimizer: cf.Args(\n      type_=\"huggingface_adamw\",\n      lr=3e-5,\n      correct_bias=True\n  )\n  train.Trainer.__init__: cf.Args(\n      num_epochs=3,\n      learning_rate_scheduler=\"polynomial_decay\",\n      grad_norm=1.0,\n  )\n})\n```\n\nWith this approach, this configuration file will make it clear which (hyper)parameters are set and\nwhere. This improves overall readability of the configuration file.\n\n🐍 Last but not least, the configuration file is written in Python, you can use variables, lambdas,\netc to further modularize.\n\n### Configuring via the command line\n\nIn case you want to change one variable at a time, this package supports configuration from the\ncommand line, for example:\n\n```console\npython example.py --sorted='Args(reverse=True)'\n```\n\n```python\nimport sys\nimport config as cf\n\ncf.add(cf.parse_cli_args(sys.argv[1:]))\n```\n\n### Logging the configuration\n\nLastly, it's useful to track the configuration file by logging it. This package supports that\nvia `config.log`. In the example below, we log the configuration to\n[Comet](https://www.comet.ml/).\n\n```python\nfrom comet_ml import Experiment\nimport config as cf\n\nexperiment = Experiment()\nexperiment.log_parameters(cf.log())\n```\n\n### Advanced: Sharing configurations between processes\n\nIn multiprocessing, it may be useful to share the configuration file between processes. In this\ncase, the configuration can be exported to another process and then subsequently imported, see\nbelow:\n\n```python\nfrom multiprocessing import Process\nimport config as cf\n\ndef handler(configs: cf.Config):\n    cf.add(configs)\n\nif __name__ == \"__main__\":\n    process = Process(target=handler, args=(cf.export(),))\n    process.start()\n    process.join()\n```\n\n### Advanced: Ensuring the configuration is used\n\nIn a large code base, it might be hard to tell if the configuration has been set for every function\ncall. In this case, we've exposed `config.trace` which can double check every function call\nagainst the configuration, see below:\n\n```python\nimport sys\nimport config as cf\n\ndef configured(a=111):\n    pass\n\nsys.settrace(cf.trace)\ncf.add({configured: cf.Args(a=1)})\n\nconfigured()  # `cf.trace` issues a WARNING!\nconfigured(a=cf.get())\n```\n\nWe also have another option for faster tracing with `config.enable_fast_trace`. Instead of a system\nwide trace, this traces the configured functions by modifying their code and inserting a trace\nfunction at the beginning of the function definition. This has a MUCH lower overhead; however, it is\nstill in beta due to the number of edge cases.\n\n### Advanced: Find unused configurations\n\nIn a large code base, you may have a lot of configurations, some of which are no longer being used.\n`purge` can be run on a process exit, and it'll warn you if configurations were not used.\n\n```python\nimport atexit\nimport config as cf\n\natexit.register(cf.purge)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPetrochukM%2FHParams","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPetrochukM%2FHParams","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPetrochukM%2FHParams/lists"}