{"id":13418337,"url":"https://github.com/google/gin-config","last_synced_at":"2025-05-10T05:07:15.742Z","repository":{"id":37502453,"uuid":"138929098","full_name":"google/gin-config","owner":"google","description":"Gin provides a lightweight configuration framework for Python","archived":false,"fork":false,"pushed_at":"2025-04-30T17:01:01.000Z","size":474,"stargazers_count":2099,"open_issues_count":60,"forks_count":115,"subscribers_count":19,"default_branch":"master","last_synced_at":"2025-05-08T20:57:51.571Z","etag":null,"topics":["configuration-management","python","tensorflow","tensorflow-experiments"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/google.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-06-27T20:21:12.000Z","updated_at":"2025-05-08T11:43:52.000Z","dependencies_parsed_at":"2024-06-03T12:43:00.553Z","dependency_job_id":"acd00c96-6b9a-43fd-9f7f-96c5a413ffaa","html_url":"https://github.com/google/gin-config","commit_stats":{"total_commits":155,"total_committers":24,"mean_commits":6.458333333333333,"dds":0.6838709677419355,"last_synced_commit":"88915e454f58b3b0eef55b6dc6bebabf3db7a5b7"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fgin-config","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fgin-config/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fgin-config/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/google%2Fgin-config/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/google","download_url":"https://codeload.github.com/google/gin-config/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253157476,"owners_count":21863127,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["configuration-management","python","tensorflow","tensorflow-experiments"],"created_at":"2024-07-30T22:01:01.164Z","updated_at":"2025-05-08T22:30:23.394Z","avatar_url":"https://github.com/google.png","language":"Python","readme":"# Gin Config\n\n\n**Authors**: Dan Holtmann-Rice, Sergio Guadarrama, Nathan Silberman\n**Contributors**: Oscar Ramirez, Marek Fiser\n\n\u003c!----\u003e\n\nGin provides a lightweight configuration framework for Python, based on\ndependency injection. Functions or classes can be decorated with\n`@gin.configurable`, allowing default parameter values to be supplied from a\nconfig file (or passed via the command line) using a simple but powerful syntax.\nThis removes the need to define and maintain configuration objects (e.g.\nprotos), or write boilerplate parameter plumbing and factory code, while often\ndramatically expanding a project's flexibility and configurability.\n\nGin is particularly well suited for machine learning experiments (e.g. using\nTensorFlow), which tend to have many parameters, often nested in complex ways.\n\nThis is not an official Google product.\n\n## Table of Contents\n\n[TOC]\n\n## Basic usage\n\nThis section provides a high-level overview of Gin's main features, ordered\nroughly from \"basic\" to \"advanced\". More details on these and other features can\nbe found in the [user guide].\n\n[user guide]: https://github.com/google/gin-config/tree/master/docs/index.md\n\n### 1. Setup\n\n\nInstall Gin with pip:\n\n```shell\npip install gin-config\n```\n\nInstall Gin from source:\n\n```shell\ngit clone https://github.com/google/gin-config\ncd gin-config\npython -m setup.py install\n```\n\nImport Gin (without TensorFlow functionality):\n\n```python\nimport gin\n```\n\nImport additional TensorFlow-specific functionality via the `gin.tf` module:\n\n```python\nimport gin.tf\n```\n\nImport additional PyTorch-specific functionality via the `gin.torch` module:\n\n```python\nimport gin.torch\n```\n\n\n### 2. Configuring default values with Gin (`@gin.configurable` and \"bindings\")\n\nAt its most basic, Gin can be seen as a way of providing or changing default\nvalues for function or constructor parameters. To make a function's parameters\n\"configurable\", Gin provides the `gin.configurable` decorator:\n\n```python\n@gin.configurable\ndef dnn(inputs,\n        num_outputs,\n        layer_sizes=(512, 512),\n        activation_fn=tf.nn.relu):\n  ...\n```\n\nThis decorator registers the `dnn` function with Gin, and automatically makes\nall of its parameters configurable. To set (\"bind\") a value for the\n`layer_sizes` parameter above within a \".gin\" configuration file:\n\n```python\n# Inside \"config.gin\"\ndnn.layer_sizes = (1024, 512, 128)\n```\n\nBindings have syntax `function_name.parameter_name = value`. All Python literal\nvalues are supported as `value` (numbers, strings, lists, tuples, dicts). Once\nthe config file has been parsed by Gin, any future calls to `dnn` will use the\nGin-specified value for `layer_sizes` (unless a value is explicitly provided by\nthe caller).\n\nClasses can also be marked as configurable, in which case the configuration\napplies to constructor parameters:\n\n```python\n@gin.configurable\nclass DNN(object):\n  # Constructor parameters become configurable.\n  def __init__(self,\n               num_outputs,\n               layer_sizes=(512, 512),\n               activation_fn=tf.nn.relu):\n    ...\n\n  def __call__(inputs):\n    ...\n```\n\nWithin a config file, the class name is used when binding values to constructor\nparameters:\n\n```python\n# Inside \"config.gin\"\nDNN.layer_sizes = (1024, 512, 128)\n```\n\nFinally, after defining or importing all configurable classes or functions,\nparse your config file to bind your configurations (to also permit multiple\nconfig files and command line overrides, see\n[`gin.parse_config_files_and_bindings`][multiple files]):\n\n```python\ngin.parse_config_file('config.gin')\n```\n\nNote that no other changes are required to the Python code, beyond adding the\n`gin.configurable` decorator and a call to one of Gin's parsing functions.\n\n[multiple files]: https://github.com/google/gin-config/tree/master/docs/index.md#experiments-with-multiple-gin-files-and-extra-command-line-bindings\n\n### 3. Passing functions, classes, and instances (\"configurable references\")\n\nIn addition to accepting Python literal values, Gin also supports passing other\nGin-configurable functions or classes. In the example above, we might want to\nchange the `activation_fn` parameter. If we have registered, say `tf.nn.tanh`\nwith Gin (see [registering external functions][external configurables]), we can\npass it to `activation_fn` by referring to it as `@tanh` (or `@tf.nn.tanh`):\n\n```python\n# Inside \"config.gin\"\ndnn.activation_fn = @tf.nn.tanh\n```\n\nGin refers to `@name` constructs as *configurable references*. Configurable\nreferences work for classes as well:\n\n```python\ndef train_fn(..., optimizer_cls, learning_rate):\n  optimizer = optimizer_cls(learning_rate)\n  ...\n```\n\nThen, within a config file:\n\n```python\n# Inside \"config.gin\"\ntrain_fn.optimizer_cls = @tf.train.GradientDescentOptimizer\n...\n```\n\nSometimes it is necessary to pass the result of calling a specific function or\nclass constructor. Gin supports \"evaluating\" configurable references via the\n`@name()` syntax. For example, say we wanted to use the class form of `DNN` from\nabove (which implements `__call__` to \"behave\" like a function) in the following\nPython code:\n\n```python\ndef build_model(inputs, network_fn, ...):\n  logits = network_fn(inputs)\n  ...\n```\n\nWe could pass an instance of the `DNN` class to the `network_fn` parameter:\n\n```python\n# Inside \"config.gin\"\nbuild_model.network_fn = @DNN()\n```\n\nTo use evaluated references, all of the referenced function or class's\nparameters must be provided via Gin. The call to the function or constructor\ntakes place *just before* the call to the function to which the result is\npassed, In the above example, this would be just before `build_model` is called.\n\nThe result is not cached, so a new `DNN` instance will be constructed for each\ncall to `build_model`.\n\n[external configurables]: https://github.com/google/gin-config/tree/master/docs/index.md#making-existing-classes-or-functions-configurable\n\n### 4. Configuring the same function in different ways (\"scopes\")\n\nWhat happens if we want to configure the same function in different ways? For\ninstance, imagine we're building a GAN, where we might have a \"generator\"\nnetwork and a \"discriminator\" network. We'd like to use the `dnn` function above\nto construct both, but with different parameters:\n\n```python\ndef build_model(inputs, generator_network_fn, discriminator_network_fn, ...):\n  ...\n```\n\nTo handle this case, Gin provides \"scopes\", which provide a name for a specific\nset of bindings for a given function or class. In both bindings and references,\nthe \"scope name\" precedes the function name, separated by a \"`/`\" (i.e.,\n`scope_name/function_name`):\n\n```python\n# Inside \"config.gin\"\nbuild_model.generator_network_fn = @generator/dnn\nbuild_model.discriminator_network_fn = @discriminator/dnn\n\ngenerator/dnn.layer_sizes = (128, 256)\ngenerator/dnn.num_outputs = 784\n\ndiscriminator/dnn.layer_sizes = (512, 256)\ndiscriminator/dnn.num_outputs = 1\n\ndnn.activation_fn = @tf.nn.tanh\n```\n\nIn this example, the generator network has increasing layer widths and 784\noutputs, while the discriminator network has decreasing layer widths and 1\noutput.\n\nAny parameters set on the \"root\" (unscoped) function name are inherited by\nscoped variants (unless explicitly overridden), so in the above example both the\ngenerator and the discriminator use the `tf.nn.tanh` activation function.  This\nworks in general for a hierarchy of scopes, e.g., if we are in a scope named\n`a/b` the config will inherit all the values from scope `a`.\n\n### 5. Full hierarchical configuration {#full-hierarchical}\n\nThe greatest degree of flexibility and configurability in a project is achieved\nby writing small modular functions and \"wiring them up\" hierarchically via\n(possibly scoped) references. For example, this code sketches a generic training\nsetup that could be used with the `tf.estimator.Estimator` API:\n\n```python\n@gin.configurable\ndef build_model_fn(network_fn, loss_fn, optimize_loss_fn):\n  def model_fn(features, labels):\n    logits = network_fn(features)\n    loss = loss_fn(labels, logits)\n    train_op = optimize_loss_fn(loss)\n    ...\n  return model_fn\n\n@gin.configurable\ndef optimize_loss(loss, optimizer_cls, learning_rate):\n  optimizer = optimizer_cls(learning_rate)\n  return optimizer.minimize(loss)\n\n@gin.configurable\ndef input_fn(file_pattern, batch_size, ...):\n  ...\n\n@gin.configurable\ndef run_training(train_input_fn, eval_input_fn, estimator, steps=1000):\n  estimator.train(train_input_fn, steps=steps)\n  estimator.evaluate(eval_input_fn)\n  ...\n```\n\nIn conjunction with suitable [external configurables] to register TensorFlow\nfunctions/classes (e.g., `Estimator` and various optimizers), this could be\nconfigured as follows:\n\n```python\n# Inside \"config.gin\"\nrun_training.train_input_fn = @train/input_fn\nrun_training.eval_input_fn = @eval/input_fn\n\ninput_fn.batch_size = 64  # Shared by both train and eval...\ntrain/input_fn.file_pattern = ...\neval/input_fn.file_pattern = ...\n\n\nrun_training.estimator = @tf.estimator.Estimator()\ntf.estimator.Estimator.model_fn = @build_model_fn()\n\nbuild_model_fn.network_fn = @dnn\ndnn.layer_sizes = (1024, 512, 256)\n\nbuild_model_fn.loss_fn = @tf.losses.sparse_softmax_cross_entropy\n\nbuild_model_fn.optimize_loss_fn = @optimize_loss\n\noptimize_loss.optimizer_cls = @tf.train.MomentumOptimizer\nMomentumOptimizer.momentum = 0.9\n\noptimize_loss.learning_rate = 0.01\n```\n\nNote that it is straightforward to switch between different network functions,\noptimizers, datasets, loss functions, etc. via different config files.\n\n### 6. Additional features\n\nAdditional features described in more detail in the [user guide] include:\n\n-   Automatic logging of all configured parameter values (the [\"operative\n    config\"][operative config]), including [TensorBoard integration].\n-   [\"Macros\"][macros], to specify a value used in multiple\n    places within a config, as well as Python-defined constants.\n-   [Module imports][imports] and [config file inclusion][includes].\n-   [Disambiguation][modules] of configurable names via modules.\n\n[operative config]: https://github.com/google/gin-config/tree/master/docs/index.md#retrieving-operative-parameter-values\n[macros]: https://github.com/google/gin-config/tree/master/docs/index.md#gin-macros\n[imports]: https://github.com/google/gin-config/tree/master/docs/index.md#importing-modules-from-within-a-gin-file\n[includes]: https://github.com/google/gin-config/tree/master/docs/index.md#including-other-gin-files\n[TensorBoard integration]: https://github.com/google/gin-config/tree/master/docs/index.md#saving-gins-operative-config-to-a-file-and-tensorboard\n[modules]: https://github.com/google/gin-config/tree/master/docs/index.md#handling-naming-collisions-with-modules\n\n## Best practices\n\nAt a high level, we recommend using the minimal feature set required to achieve\nyour project's desired degree of configurability. Many projects may only\nrequire the features outlined in sections 2 or 3 above. Extreme configurability\ncomes at some cost to understandability, and the tradeoff should be carefully\nevaluated for a given project.\n\nGin is still in alpha development and some corner-case behaviors may be\nchanged in backwards-incompatible ways. We recommend the following best\npractices:\n\n-   Minimize use of evaluated configurable references (`@name()`), especially\n    when combined with macros (where the fact that the value is not cached may\n    be surprising to new users).\n-   Avoid nesting of scopes (i.e., `scope1/scope2/function_name`). While\n    supported there is some ongoing debate around ordering and behavior.\n-   When passing an unscoped reference (`@name`) as a parameter of a scoped\n    function (`some_scope/fn.param`), the unscoped reference gets called in the\n    scope of the function it is passed to... but don't rely on this behavior.\n-   Wherever possible, prefer to use a function or class's name as its\n    configurable name, instead of overriding it. In case of naming collisions,\n    use module names (which are encouraged to be renamed to match common usage)\n    for disambiguation.\n-   In fact, to aid readability for complex config files, we gently suggest\n    always including module names to help make it easier to find corresponding\n    definitions in Python code.\n-   When doing [\"full hierarchical configuration\"](#full-hierarchical), structure\n    the code to minimize the number of \"top-level\" functions that are\n    configured without themselves being passed as parameters. In other words,\n    the configuration tree should have only one root.\n\nIn short, use Gin responsibly :)\n\n## Syntax quick reference\n\nA quick reference for syntax unique to Gin (which otherwise supports\nnon-control-flow Python syntax, including literal values and line\ncontinuations). Note that where function and class names are used, these may\ninclude a dotted module name prefix (`some.module.function_name`).\n\n\u003ctable\u003e\n  \u003cthead\u003e\n    \u003ctr\u003e\n      \u003cth\u003eSyntax\u003c/th\u003e\n      \u003cth\u003eDescription\u003c/th\u003e\n    \u003c/tr\u003e\n  \u003c/thead\u003e\n\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e@gin.configurable\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eDecorator in Python code that registers a function or class with Gin,\n        wrapping/replacing it with a \"configurable\" version that respects Gin\n        parameter overrides. A function or class annotated with\n        `@gin.configurable` will have its parameters overridden by any provided\n        configs even when called directly from other Python code.\n      .\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e@gin.register\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eDecorator in Python code that only registers a function or class with\n        Gin, but does *not* replace it with its \"configurable\" version.\n        Functions or classes annotated with `@gin.register` will *not* have\n        their parameters overridden by Gin configs when called directly from\n        other Python code. However, any references in config strings or files to\n        these functions (`@some_name` syntax, see below) will apply any provided\n        configuration.\n      \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003ename.param\u0026nbsp;=\u0026nbsp;value\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eBasic syntax of a Gin binding. Once this is parsed, when the\n      function or class named \u003ccode\u003ename\u003c/code\u003e is called, it will receive\n      \u003ccode\u003evalue\u003c/code\u003e as the value for \u003ccode\u003eparam\u003c/code\u003e, unless a\n      value is explicitly supplied by the caller. Any Python literal may be\n      supplied as \u003ccode\u003evalue\u003c/code\u003e.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e@some_name\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eA \u003cem\u003ereference\u003c/em\u003e to another function or class named\n      \u003ccode\u003esome_name\u003c/code\u003e. This may be given as the value of a binding, to\n      supply function- or class-valued parameters.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e@some_name()\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eAn \u003cem\u003eevaluated reference\u003c/em\u003e. Instead of supplying the function\n      or class directly, the result of calling \u003ccode\u003esome_name\u003c/code\u003e is\n      passed instead. Note that the result is not cached; it is recomputed\n      each time it is required.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003escope/name.param\u0026nbsp;=\u0026nbsp;value\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eA scoped binding. The binding is only active when \u003ccode\u003ename\u003c/code\u003e\n      is called within scope \u003ccode\u003escope\u003c/code\u003e.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e@scope/some_name\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eA scoped reference. When this is called, the call will be within\n      scope \u003ccode\u003escope\u003c/code\u003e, applying any relevant scoped bindings.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003eMACRO_NAME\u0026nbsp;=\u0026nbsp;value\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eA macro. This provides a shorthand name for the expression on the\n      right hand side.\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n      \u003ctd\u003e\u003ccode\u003e%MACRO_NAME\u003c/code\u003e\u003c/td\u003e\n      \u003ctd\u003eA reference to the macro \u003ccode\u003eMACRO_NAME\u003c/code\u003e. This has the\n      effect of textually replacing \u003ccode\u003e%MACRO_NAME\u003c/code\u003e with whatever\n      expression it was associated with. Note in particular that the result\n      of evaluated references are not cached.\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n","funding_links":[],"categories":["Multipurpose","Python","Running","Desktop App Development","Configuration"],"sub_categories":["Python Toolkit"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fgin-config","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgoogle%2Fgin-config","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgoogle%2Fgin-config/lists"}