{"id":40855151,"url":"https://github.com/offscale/cdd-python","last_synced_at":"2026-01-21T23:45:01.022Z","repository":{"id":41805956,"uuid":"279060109","full_name":"offscale/cdd-python","owner":"offscale","description":"Open API to/fro routes, models, and tests. Convert between docstrings, classes, methods, argparse, SQLalchemy, Pydantic, JSON-schema.","archived":false,"fork":false,"pushed_at":"2025-07-29T00:50:48.000Z","size":2033,"stargazers_count":17,"open_issues_count":1,"forks_count":5,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-09-29T08:10:40.986Z","etag":null,"topics":["argparse","compiler","docstrings","openapi","sqlalchemy"],"latest_commit_sha":null,"homepage":"https://compilers.com.au","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/offscale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-07-12T12:30:09.000Z","updated_at":"2025-07-29T00:50:52.000Z","dependencies_parsed_at":"2022-07-08T03:01:35.168Z","dependency_job_id":"c6934180-fcc4-4025-a4d6-2807b35fa819","html_url":"https://github.com/offscale/cdd-python","commit_stats":{"total_commits":501,"total_committers":3,"mean_commits":167.0,"dds":0.007984031936127733,"last_synced_commit":"d6226dff97779db59b6e3f3ca3c492a0c9192239"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/offscale/cdd-python","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/offscale%2Fcdd-python","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/offscale%2Fcdd-python/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/offscale%2Fcdd-python/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/offscale%2Fcdd-python/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/offscale","download_url":"https://codeload.github.com/offscale/cdd-python/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/offscale%2Fcdd-python/sbom","scorecard":{"id":703029,"data":{"date":"2025-08-11","repo":{"name":"github.com/offscale/cdd-python","commit":"1a03e6c92fb59ff111f9a4c94814dc6681ef33ec"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.2,"checks":[{"name":"Maintained","score":4,"reason":"5 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 4","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Warn: no topLevel permission defined: .github/workflows/main.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE-APACHE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE-APACHE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/offscale/cdd-python/main.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:32: update your workflow using https://app.stepsecurity.io/secureworkflow/offscale/cdd-python/main.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:70: update your workflow using https://app.stepsecurity.io/secureworkflow/offscale/cdd-python/main.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/main.yml:72: update your workflow using https://app.stepsecurity.io/secureworkflow/offscale/cdd-python/main.yml/master?enable=pin","Warn: containerImage not pinned by hash: Dockerfile:1: pin your Docker image by updating python:3.6-alpine to python:3.6-alpine@sha256:579978dec4602646fe1262f02b96371779bfb0294e92c91392707fa999c0c989","Warn: pipCommand not pinned by hash: Dockerfile:6-8","Warn: pipCommand not pinned by hash: Dockerfile:6-8","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:38","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:39","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:40","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:46","Warn: downloadThenRun not pinned by hash: .github/workflows/main.yml:49","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:78","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:79","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:80","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:87","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:92","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:97","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:115","Warn: downloadThenRun not pinned by hash: .github/workflows/main.yml:118","Warn: pipCommand not pinned by hash: .github/workflows/main.yml:127","Info: 0 out of 4 GitHub-owned GitHubAction dependencies pinned","Info: 0 out of 1 containerImage dependencies pinned","Info: 0 out of 14 pipCommand dependencies pinned","Info: 0 out of 2 downloadThenRun dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Vulnerabilities","score":3,"reason":"7 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: PYSEC-2024-48 / GHSA-fj7x-q9j7-g6q6","Warn: Project is vulnerable to: PYSEC-2021-142 / GHSA-8q59-q68h-6hv4","Warn: Project is vulnerable to: PYSEC-2018-49 / GHSA-rprw-h62v-c2w7","Warn: Project is vulnerable to: PYSEC-2013-22 / GHSA-27x4-j476-jp5f","Warn: Project is vulnerable to: PYSEC-2025-49 / GHSA-5rjg-fvgr-3xxf","Warn: Project is vulnerable to: GHSA-cx63-2mw6-8hw5","Warn: Project is vulnerable to: PYSEC-2022-43012 / GHSA-r9hx-vwmv-q579"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-22T05:40:20.722Z","repository_id":41805956,"created_at":"2025-08-22T05:40:20.722Z","updated_at":"2025-08-22T05:40:20.722Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28646966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T21:29:11.980Z","status":"ssl_error","status_checked_at":"2026-01-21T21:24:31.872Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["argparse","compiler","docstrings","openapi","sqlalchemy"],"created_at":"2026-01-21T23:45:00.196Z","updated_at":"2026-01-21T23:45:01.008Z","avatar_url":"https://github.com/offscale.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"cdd-python\n==========\n![Python version range](https://img.shields.io/badge/python-3.6%20|%203.7%20|%203.8%20|%203.9%20|%203.10%20|%203.11%20|%203.12%20|%203.13-blue.svg)\n![Python implementation](https://img.shields.io/badge/implementation-cpython-blue.svg)\n[![License](https://img.shields.io/badge/license-Apache--2.0%20OR%20MIT-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n[![Linting, testing, coverage, and release](https://github.com/offscale/cdd-python/workflows/Linting,%20testing,%20coverage,%20and%20release/badge.svg)](https://github.com/offscale/cdd-python/actions)\n![Tested OSs, others may work](https://img.shields.io/badge/Tested%20on-Linux%20|%20macOS%20|%20Windows-green)\n![Documentation coverage](https://raw.githubusercontent.com/offscale/cdd-python/master/.github/doccoverage.svg)\n[![codecov](https://codecov.io/gh/offscale/cdd-python/graph/badge.svg?token=TndsFusENZ)](https://codecov.io/gh/offscale/cdd-python)\n[![black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n[![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat\u0026labelColor=ef8336)](https://pycqa.github.io/isort)\n[![PyPi: release](https://img.shields.io/pypi/v/python-cdd.svg?maxAge=3600)](https://pypi.org/project/python-cdd)\n[![hosted documentation](https://img.shields.io/badge/hosted-docs-white)](https://offscale.io/cdd-python/)\n\n[OpenAPI](https://openapis.org) to/fro routes, models, and tests. Convert between docstrings, `class`es,\nmethods, [argparse](https://docs.python.org/3/library/argparse.html), pydantic,\nand [SQLalchemy](https://sqlalchemy.org).\n\nPublic SDK works with filenames, source code, and even in memory constructs (e.g., as imported into your REPL).\n\n## Features\n\n| Type | Parse | Emit | Convert to all other Types |\n|---------------------------------------------------------------------------------------------------------------------------------------------------------|-------|------|----------------------------|\n| docstrings (between Google, NumPy, ReST formats; and betwixt type annotations and docstring) | ✅ | ✅ | ✅ |\n| `class`es | ✅ | ✅ | ✅ |\n| functions | ✅ | ✅ | ✅ |\n| [`argparse` CLI generating](https://docs.python.org/3/library/argparse.html#argumentparser-objects) functions | ✅ | ✅ | ✅ |\n| JSON-schema | ✅ | ✅ | ✅ |\n| [SQLalchemy `class`es](https://docs.sqlalchemy.org/en/14/orm/mapping_styles.html#orm-declarative-mapping) | ✅ | ✅ | ✅ |\n| [SQLalchemy `Table`s](https://docs.sqlalchemy.org/en/14/core/metadata.html#sqlalchemy.schema.Table) | ✅ | ✅ | ✅ |\n| [SQLalchemy hybrid `class`es](https://docs.sqlalchemy.org/en/14/orm/declarative_tables.html#declarative-with-imperative-table-a-k-a-hybrid-declarative) | ✅ | ✅ | ✅ |\n| [pydantic `class`es](https://pydantic-docs.helpmanual.io/usage/schema/) | ✅ | ✅ | ✅ |\n\n### [OpenAPI](https://openapis.org) composite\n\nThe [OpenAPI](https://swagger.io/specification/) parser and emitter\nutilises:\n\n| Type | Parse | Emit |\n|-----------------------------------------------------------------------------------------------------------|-------|------|\n| [Bottle route functions](https://bottlepy.org/docs/dev/api.html#routing) | WiP | WiP |\n| [FastAPI route functions](https://fastapi.tiangolo.com/tutorial/body/#request-body-path-query-parameters) | ✅ | ❌ |\n| JSON-schema (e.g., from [SQLalchemy](https://docs.sqlalchemy.org)) | ✅ | ✅ |\n\n## Install package\n\n### PyPi\n\n python -m pip install python-cdd\n\n### Master\n\n python -m pip install -r https://raw.githubusercontent.com/offscale/cdd-python/master/requirements.txt\n python -m pip install https://api.github.com/repos/offscale/cdd-python/zipball#egg=cdd\n\n## Goal\n\nEasily create and maintain Database / ORM models and REST APIs out of existing Python SDKs.\n\nFor example, this can be used to expose TensorFlow in a REST API and store its parameters in an SQL database.\n\n## Relation to other projects\n\nThis was created to aid in the `ml_params` project. It exposes an `@abstractclass` which is implemented [officially] by\nmore than 8 projects.\n\nDue to the nature of ML frameworks, `ml_params`' `def train(self, \u003cthese\u003e)` has a potentially large number of arguments.\nAccumulate the complexity of maintaining interfaces as the underlying release changes (e.g, new version of PyTorch),\nadd in the extra interfaces folks find useful (CLIs, REST APIs, SQL models, \u0026etc.); and you end up needing a team to\nmaintain it.\n\nThat's unacceptable. The only existing solutions maintainable by one engineer involve dynamic generation,\nwith no static, editable interfaces available. This means developer tooling becomes useless for debugging,\nintrospection, and documentation.\n\nTo break it down, with current tooling there is no way to know:\n\n- What arguments can be provided to `train`\n- What CLI arguments are available\n- What 'shape' the `Config` takes\n\nSome of these problems can be solved dynamically, however in doing so one loses developer-tool insights.\nThere is no code-completion, and likely the CLI parser won't provide you with the enumeration of possibilities.\n\nFor migration from Google App Engine, this project builds upon cdd-python for a unidirectional\nexperience: https://github.com/offscale/cdd-python-gae\n\nAlso, a proof-of-concept was made\nto [generate documentation using cdd-python](https://github.com/SamuelMarks/griffe/tree/rest) (`mkdocs` core\nreplacement).\n\n## SDK example (REPL)\n\nTo create a `class`\nfrom [`tf.keras.optimizers.Adam`](https://tensorflow.org/api_docs/python/tf/keras/optimizers/Adam):\n\n```python\n\u003e\u003e \u003e from cdd.shared.source_transformer import to_code\n\n\u003e\u003e \u003e import cdd.class_.emit\n\n\u003e\u003e \u003e import cdd.class_.parse\n\n\u003e\u003e \u003e import tensorflow as tf\n\n\u003e\u003e \u003e from typing import Optional\n\n\u003e\u003e \u003e print(to_code(cdd.class_.emit.class_(cdd.class_.parse.class_(\n tf.keras.optimizers.Adam,\n merge_inner_function=\"__init__\"\n),\n class_name=\"AdamConfig\")))\n\n\nclass AdamConfig(object):\n \"\"\"\n Optimizer that implements the Adam algorithm.\n\n Adam optimization is a stochastic gradient descent method that is based on\n adaptive estimation of first-order and second-order moments.\n\n According to\n [Kingma et al., 2014](http://arxiv.org/abs/1412.6980),\n the method is \"*computationally\n efficient, has little memory requirement, invariant to diagonal rescaling of\n gradients, and is well suited for problems that are large in terms of\n data/parameters*\".\n\n\n Usage:\n\n \u003e\u003e\u003e opt = tf.keras.optimizers.Adam(learning_rate=0.1)\n \u003e\u003e\u003e var1 = tf.Variable(10.0)\n \u003e\u003e\u003e loss = lambda: (var1 ** 2)/2.0 # d(loss)/d(var1) == var1\n \u003e\u003e\u003e step_count = opt.minimize(loss, [var1]).numpy()\n \u003e\u003e\u003e # The first step is `-learning_rate*sign(grad)`\n \u003e\u003e\u003e var1.numpy()\n 9.9\n\n Reference:\n - [Kingma et al., 2014](http://arxiv.org/abs/1412.6980)\n - [Reddi et al., 2018](\n https://openreview.net/pdf?id=ryQu7f-RZ) for `amsgrad`.\n\n Notes:\n\n The default value of 1e-7 for epsilon might not be a good default in\n general. For example, when training an Inception network on ImageNet a\n current good choice is 1.0 or 0.1. Note that since Adam uses the\n formulation just before Section 2.1 of the Kingma and Ba paper rather than\n the formulation in Algorithm 1, the \"epsilon\" referred to here is \"epsilon\n hat\" in the paper.\n\n The sparse implementation of this algorithm (used when the gradient is an\n IndexedSlices object, typically because of `tf.gather` or an embedding\n lookup in the forward pass) does apply momentum to variable slices even if\n they were not used in the forward pass (meaning they have a gradient equal\n to zero). Momentum decay (beta1) is also applied to the entire momentum\n accumulator. This means that the sparse behavior is equivalent to the dense\n behavior (in contrast to some momentum implementations which ignore momentum\n unless a variable slice was actually used).\n\n :cvar learning_rate: A `Tensor`, floating point value, or a schedule that is a\n `tf.keras.optimizers.schedules.LearningRateSchedule`, or a callable that takes no arguments and\n returns the actual value to use, The learning rate.\n :cvar beta_1: A float value or a constant float tensor, or a callable that takes no arguments and\n returns the actual value to use. The exponential decay rate for the 1st moment estimates.\n :cvar beta_2: A float value or a constant float tensor, or a callable that takes no arguments and\n returns the actual value to use, The exponential decay rate for the 2nd moment estimates.\n :cvar epsilon: A small constant for numerical stability. This epsilon is \"epsilon hat\" in the\n Kingma and Ba paper (in the formula just before Section 2.1), not the epsilon in Algorithm 1 of the\n paper.\n :cvar amsgrad: Boolean. Whether to apply AMSGrad variant of this algorithm from the paper \"On the\n Convergence of Adam and beyond\".\n :cvar name: Optional name for the operations created when applying gradients.\n :cvar kwargs: Keyword arguments. Allowed to be one of `\"clipnorm\"` or `\"clipvalue\"`. `\"clipnorm\"`\n (float) clips gradients by norm; `\"clipvalue\"` (float) clips gradients by value.\"\"\"\n learning_rate: float = 0.001\n beta_1: float = 0.9\n beta_2: float = 0.999\n epsilon: float = 1e-07\n amsgrad: bool = False\n name: Optional[str] = 'Adam'\n kwargs: Optional[dict] = None\n _HAS_AGGREGATE_GRAD: bool = True\n```\n\n### Approach\n\nTraverse the AST, and emit the modifications, such that each \"format\" can convert to each other.\nType asymmetries are added to the docstrings, e.g., \"primary_key\" has no equivalent in a regular python func argument,\nso is added as `\":param my_id: [PK] The unique identifier\"`.\n\nThe following are the different formats supported, all of which can convert betwixt each-other:\n\n#### Docstring\n\n```reStructuredText\nAcquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare library\n\n:param dataset_name: name of dataset. Defaults to mnist\n:type dataset_name: ```str```\n\n:param tfds_dir: directory to look for models in. Defaults to ~/tensorflow_datasets\n:type tfds_dir: ```Optional[str]```\n\n:param K: backend engine, e.g., `np` or `tf`. Defaults to np\n:type K: ```Union[np, tf]```\n\n:param as_numpy: Convert to numpy ndarrays\n:type as_numpy: ```Optional[bool]```\n\n:param data_loader_kwargs: pass this as arguments to data_loader function\n:type data_loader_kwargs: ```**data_loader_kwargs```\n\n:return: Train and tests dataset splits. Defaults to (np.empty(0), np.empty(0))\n:rtype: ```Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]]```\n```\n\n##### `class`\n\n```python\nfrom typing import Optional, Union, Tuple, Literal\n\nimport numpy as np\nimport tensorflow as tf\n\n\nclass TargetClass(object):\n \"\"\"\n Acquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare library\n\n :cvar dataset_name: name of dataset. Defaults to mnist\n :cvar tfds_dir: directory to look for models in. Defaults to ~/tensorflow_datasets\n :cvar K: backend engine, e.g., `np` or `tf`. Defaults to np\n :cvar as_numpy: Convert to numpy ndarrays\n :cvar data_loader_kwargs: pass this as arguments to data_loader function\n :cvar return_type: Train and tests dataset splits. Defaults to (np.empty(0), np.empty(0))\"\"\"\n\n dataset_name: str = 'mnist'\n tfds_dir: Optional[str] = '~/tensorflow_datasets'\n K: Literal['np', 'tf'] = 'np'\n as_numpy: Optional[bool] = None\n data_loader_kwargs: dict = {}\n return_type: Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]] = (\n np.empty(0),\n np.empty(0),\n )\n```\n\n##### `class` method\n\n```python\nfrom typing import Optional, Union, Tuple, Literal\n\nimport numpy as np\nimport tensorflow as tf\n\n\nclass C(object):\n \"\"\" C class (mocked!) \"\"\"\n\n def method_name(\n self,\n dataset_name: str = 'mnist',\n tfds_dir: Optional[str] = '~/tensorflow_datasets',\n K: Literal['np', 'tf'] = 'np',\n as_numpy: Optional[bool] = None,\n **data_loader_kwargs\n ) -\u003e Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]]:\n \"\"\"\n Acquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare library\n \n :param dataset_name: name of dataset.\n \n :param tfds_dir: directory to look for models in.\n \n :param K: backend engine, e.g., `np` or `tf`.\n \n :param as_numpy: Convert to numpy ndarrays\n \n :param data_loader_kwargs: pass this as arguments to data_loader function\n \n :return: Train and tests dataset splits.\n \"\"\"\n return np.empty(0), np.empty(0)\n```\n\n##### Argparse augmenting function\n\n```python\nfrom typing import Union, Tuple\nfrom json import loads\n\nimport numpy as np\nimport tensorflow as tf\n\n\ndef set_cli_args(argument_parser):\n \"\"\"\n Set CLI arguments\n\n :param argument_parser: argument parser\n :type argument_parser: ```ArgumentParser```\n\n :return: argument_parser, Train and tests dataset splits.\n :rtype: ```Tuple[ArgumentParser, Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]]]```\n \"\"\"\n argument_parser.description = (\n 'Acquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare library'\n )\n argument_parser.add_argument(\n '--dataset_name', type=str, help='name of dataset.', required=True, default='mnist'\n )\n argument_parser.add_argument(\n '--tfds_dir',\n type=str,\n help='directory to look for models in.',\n default='~/tensorflow_datasets',\n )\n argument_parser.add_argument(\n '--K',\n type=globals().__getitem__,\n choices=('np', 'tf'),\n help='backend engine, expr.g., `np` or `tf`.',\n required=True,\n default='np',\n )\n argument_parser.add_argument('--as_numpy', type=bool, help='Convert to numpy ndarrays')\n argument_parser.add_argument(\n '--data_loader_kwargs', type=loads, help='pass this as arguments to data_loader function'\n )\n return argument_parser, (np.empty(0), np.empty(0))\n```\n\n##### SQLalchemy\n\nThere are two variants in the latest SQLalchemy, both are supported:\n\n```py\nfrom sqlalchemy import JSON, Boolean, Column, Enum, MetaData, String, Table, create_engine\n\nengine = create_engine(\"sqlite://\", echo=True, future=True)\nmetadata = MetaData()\n\nconfig_tbl = Table(\n \"config_tbl\",\n metadata,\n Column(\n \"dataset_name\",\n String,\n doc=\"name of dataset\",\n default=\"mnist\",\n primary_key=True,\n ),\n Column(\n \"tfds_dir\",\n String,\n doc=\"directory to look for models in\",\n default=\"~/tensorflow_datasets\",\n nullable=False,\n ),\n Column(\n \"K\",\n Enum(\"np\", \"tf\", name=\"K\"),\n doc=\"backend engine, e.g., `np` or `tf`\",\n default=\"np\",\n nullable=False,\n ),\n Column(\n \"as_numpy\",\n Boolean,\n doc=\"Convert to numpy ndarrays\",\n default=None,\n nullable=True,\n ),\n Column(\n \"data_loader_kwargs\",\n JSON,\n doc=\"pass this as arguments to data_loader function\",\n default=None,\n nullable=True,\n ),\n comment='Acquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare\\n'\n '\\n'\n ':return: Train and tests dataset splits. Defaults to (np.empty(0), np.empty(0))\\n'\n ':rtype: ```Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]]```',\n)\n\nmetadata.create_all(engine)\n```\n\n```py\nfrom sqlalchemy.orm import declarative_base\nfrom sqlalchemy import JSON, Boolean, Column, Enum, String\n\nBase = declarative_base()\n\n\nclass Config(Base):\n \"\"\"\n Acquire from the official tensorflow_datasets model zoo, or the ophthalmology focussed ml-prepare\n \n :return: Train and tests dataset splits. Defaults to (np.empty(0), np.empty(0))\n :rtype: ```Union[Tuple[tf.data.Dataset, tf.data.Dataset], Tuple[np.ndarray, np.ndarray]]```\n \"\"\"\n __tablename__ = \"config_tbl\"\n\n dataset_name = Column(\n String,\n doc=\"name of dataset\",\n default=\"mnist\",\n primary_key=True,\n )\n\n tfds_dir = Column(\n String,\n doc=\"directory to look for models in\",\n default=\"~/tensorflow_datasets\",\n nullable=False,\n )\n\n K = Column(\n Enum(\"np\", \"tf\", name=\"K\"),\n doc=\"backend engine, e.g., `np` or `tf`\",\n default=\"np\",\n nullable=False,\n )\n\n as_numpy = Column(\n Boolean,\n doc=\"Convert to numpy ndarrays\",\n default=None,\n nullable=True,\n )\n\n data_loader_kwargs = Column(\n JSON,\n doc=\"pass this as arguments to data_loader function\",\n default=None,\n nullable=True,\n )\n\n def __repr__(self):\n \"\"\"\n Emit a string representation of the current instance\n \n :return: String representation of instance\n :rtype: ```str```\n \"\"\"\n\n return (\"Config(dataset_name={dataset_name!r}, tfds_dir={tfds_dir!r}, \"\n \"K={K!r}, as_numpy={as_numpy!r}, data_loader_kwargs={data_loader_kwargs!r})\").format(\n dataset_name=self.dataset_name, tfds_dir=self.tfds_dir, K=self.K,\n as_numpy=self.as_numpy, data_loader_kwargs=self.data_loader_kwargs\n )\n```\n\n## Advantages\n\n- CLI gives proper `--help` messages\n- IDE and console gives proper insights to function, and arguments, including on type\n- `class`–based interface opens this up to clean object passing\n- Rather than passing around odd ORM class entities, you can use POPO (Plain Old Python Objects) and serialise easily\n- `@abstractmethod` can add—remove, and change—as many arguments as it wants; including required arguments; without\n worry\n- Verbosity of output removes the magic. It's always clear what's going on.\n- Outputting regular code means things can be composed and extended as normally.\n\n## Disadvantages\n\n- You have to run a tool to synchronise your various formats.\n- Duplication (but the tool handles this)\n\n## Alternatives\n\n- Slow, manual duplication; or\n- Dynamic code generation, e.g., with a singular interface for everything; so everything is in one place without\n duplication\n\n## Minor other use-cases this facilitates\n\n- Switch between having types in the docstring and having the types\n inline ([PEP484](https://python.org/dev/peps/pep-0484)–style)\n- Switch between docstring formats (to/from {numpy, ReST, google})\n- Desktop GUI with wxWidgets, from the argparse layer through [Gooey](https://github.com/chriskiehl/Gooey) [one liner]\n\n## CLI for this project\n\n $ python -m cdd --help\n usage: python -m cdd [-h] [--version]\n {sync_properties,sync,gen,gen_routes,openapi,doctrans,exmod}\n ...\n\n Open API to/fro routes, models, and tests. Convert between docstrings,\n classes, methods, argparse, pydantic, and SQLalchemy.\n \n positional arguments:\n {sync_properties,sync,gen,gen_routes,openapi,doctrans,exmod}\n sync_properties Synchronise one or more properties between input and\n input_str Python files\n sync Force argparse, classes, and/or methods to be\n equivalent\n gen Generate classes, functions, argparse function,\n sqlalchemy tables and/or sqlalchemy classes from the\n input mapping\n gen_routes Generate per model route(s)\n openapi Generate OpenAPI schema from specified project(s)\n doctrans Convert docstring format of all classes and functions\n within target file\n exmod Expose module hierarchy-\u003e{functions,classes,vars} for\n parameterisation via {REST API + database,CLI,SDK}\n \n options:\n -h, --help show this help message and exit\n --version show program's version number and exit\n\n### `sync`\n\n $ python -m cdd sync --help\n usage: python -m cdd sync [-h] [--argparse-function ARGPARSE_FUNCTIONS]\n [--argparse-function-name ARGPARSE_FUNCTION_NAMES]\n [--class CLASSES] [--class-name CLASS_NAMES]\n [--function FUNCTIONS]\n [--function-name FUNCTION_NAMES] [--no-word-wrap]\n --truth\n {argparse_function,class,function,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n \n options:\n -h, --help show this help message and exit\n --argparse-function ARGPARSE_FUNCTIONS\n File where argparse function is `def`ined.\n --argparse-function-name ARGPARSE_FUNCTION_NAMES\n Name of argparse function.\n --class CLASSES File where class `class` is declared.\n --class-name CLASS_NAMES\n Name of `class`\n --function FUNCTIONS File where function is `def`ined.\n --function-name FUNCTION_NAMES\n Name of Function. If method, use Python resolution\n syntax, i.e., ClassName.function_name\n --no-word-wrap Whether word-wrap is disabled (on emission). None\n enables word-wrap. Defaults to None.\n --truth {argparse_function,class,function,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n Single source of truth. Others will be generated from\n this. Will run with first found choice.\n\n### `sync_properties`\n\n $ python -m cdd sync_properties --help\n usage: python -m cdd sync_properties [-h] --input-filename INPUT_FILENAME\n --input-param INPUT_PARAMS [--input-eval]\n --output-filename OUTPUT_FILENAME\n --output-param OUTPUT_PARAMS\n [--output-param-wrap OUTPUT_PARAM_WRAP]\n \n options:\n -h, --help show this help message and exit\n --input-filename INPUT_FILENAME\n File to find `--input-param` from\n --input-param INPUT_PARAMS\n Location within file of property. Can be top level\n like `a` for `a=5` or with the `.` syntax as in\n `--output-param`.\n --input-eval Whether to evaluate the input-param, or just leave it\n --output-filename OUTPUT_FILENAME\n Edited in place, the property within this file (to\n update) is selected by --output-param\n --output-param OUTPUT_PARAMS\n Parameter to update. E.g., `A.F` for `class A: F`,\n `f.g` for `def f(g): pass`\n --output-param-wrap OUTPUT_PARAM_WRAP\n Wrap all input_str params with this. E.g.,\n `Optional[Union[{output_param}, str]]`\n\n### `gen`\n\n $ python -m cdd gen --help\n usage: python -m cdd gen [-h] --name-tpl NAME_TPL --input-mapping\n INPUT_MAPPING [--prepend PREPEND]\n [--imports-from-file IMPORTS_FROM_FILE]\n [--parse {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table,infer}]\n --emit\n {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n -o OUTPUT_FILENAME [--emit-call]\n [--emit-and-infer-imports] [--no-word-wrap]\n [--decorator DECORATOR_LIST] [--phase PHASE]\n \n options:\n -h, --help show this help message and exit\n --name-tpl NAME_TPL Template for the name, e.g., `{name}Config`.\n --input-mapping INPUT_MAPPING\n Fully-qualified module, filepath, or directory.\n --prepend PREPEND Prepend file with this. Use '\\n' for newlines.\n --imports-from-file IMPORTS_FROM_FILE\n Extract imports from file and append to `output_file`.\n If module or other symbol path given, resolve file\n then use it.\n --parse {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table,infer}\n What type the input is.\n --emit {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n Which type to generate.\n -o OUTPUT_FILENAME, --output-filename OUTPUT_FILENAME\n Output file to write to.\n --emit-call Whether to place all the previous body into a new\n `__call__` internal function\n --emit-and-infer-imports\n Whether to emit and infer imports at the top of the\n generated code\n --no-word-wrap Whether word-wrap is disabled (on emission). None\n enables word-wrap. Defaults to None.\n --decorator DECORATOR_LIST\n List of decorators.\n --phase PHASE Which phase to run through. E.g., SQLalchemy may\n require multiple phases to resolve foreign keys.\n\nPS: If you're outputting JSON-schema and want a file per schema then:\n\n python -c 'import sys,json,os; f=open(sys.argv[1], \"rt\"); d=json.load(f); f.close(); [(lambda f: json.dump(sc,f) or f.close())(open(os.path.join(os.path.dirname(sys.argv[1]), sc[\"$id\"].rpartition(\"/\")[2]), \"wt\")) for sc in d[\"schemas\"]]' \u003cpath_to_json_file\u003e\n\n### `gen_routes`\n\n $ python -m cdd gen_routes --help\n usage: python -m cdd gen_routes [-h] --crud {CRUD,CR,C,R,U,D,CR,CU,CD,CRD}\n [--app-name APP_NAME] --model-path MODEL_PATH\n --model-name MODEL_NAME --routes-path\n ROUTES_PATH [--route ROUTE]\n \n options:\n -h, --help show this help message and exit\n --crud {CRUD,CR,C,R,U,D,CR,CU,CD,CRD}\n What of (C)reate, (R)ead, (U)pdate, (D)elete to\n generate\n --app-name APP_NAME Name of app (e.g., `app_name = Bottle();\n @app_name.get('/api') def slash(): pass`)\n --model-path MODEL_PATH\n Python module resolution (foo.models) or filepath\n (foo/models)\n --model-name MODEL_NAME\n Name of model to generate from\n --routes-path ROUTES_PATH\n Python module resolution 'foo.routes' or filepath\n 'foo/routes'\n --route ROUTE Name of the route, defaults to\n `/api/{model_name.lower()}`\n\n### `openapi`\n\n $ python -m cdd openapi --help\n usage: python -m cdd openapi [-h] [--app-name APP_NAME] --model-paths\n MODEL_PATHS --routes-paths [ROUTES_PATHS ...]\n \n options:\n -h, --help show this help message and exit\n --app-name APP_NAME Name of app (e.g., `app_name = Bottle();\n @app_name.get('/api') def slash(): pass`)\n --model-paths MODEL_PATHS\n Python module resolution (foo.models) or filepath\n (foo/models)\n --routes-paths [ROUTES_PATHS ...]\n Python module resolution 'foo.routes' or filepath\n 'foo/routes'\n\n### `doctrans`\n\n $ python -m cdd doctrans --help\n usage: python -m cdd doctrans [-h] --filename FILENAME --format\n {rest,google,numpydoc}\n (--type-annotations | --no-type-annotations | --no-word-wrap)\n \n options:\n -h, --help show this help message and exit\n --filename FILENAME Python file to convert docstrings within. Edited in\n place.\n --format {rest,google,numpydoc}\n The docstring format to replace existing format with.\n --type-annotations Inline the type, i.e., annotate PEP484 (outside\n docstring. Requires 3.6+)\n --no-type-annotations\n Ensure all types are in docstring (rather than a\n PEP484 type annotation)\n --no-word-wrap Whether word-wrap is disabled (on emission). None\n enables word-wrap. Defaults to None.\n\n### `exmod`\n\n $ python -m cdd exmod --help\n usage: python -m cdd exmod [-h] -m MODULE --emit\n {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n [--emit-sqlalchemy-submodule]\n [--extra-module [EXTRA_MODULES]] [--no-word-wrap]\n [--blacklist BLACKLIST] [--whitelist WHITELIST] -o\n OUTPUT_DIRECTORY\n [--target-module-name TARGET_MODULE_NAME] [-r]\n [--dry-run]\n \n options:\n -h, --help show this help message and exit\n -m MODULE, --module MODULE\n The module or fully-qualified name (FQN) to expose.\n --emit {argparse,class,function,json_schema,pydantic,sqlalchemy,sqlalchemy_hybrid,sqlalchemy_table}\n Which type to generate.\n --emit-sqlalchemy-submodule\n Whether to; for sqlalchemy*; emit submodule \"sqlalchem\n y_mod/{__init__,connection,create_tables}.py\"\n --extra-module [EXTRA_MODULES]\n Additional module(s) to expose; specifiable multiple\n times. Added to symbol auto-import resolver.\n --no-word-wrap Whether word-wrap is disabled (on emission). None\n enables word-wrap. Defaults to None.\n --blacklist BLACKLIST\n Modules/FQN to omit. If unspecified will emit all\n (unless whitelist).\n --whitelist WHITELIST\n Modules/FQN to emit. If unspecified will emit all\n (minus blacklist).\n -o OUTPUT_DIRECTORY, --output-directory OUTPUT_DIRECTORY\n Where to place the generated exposed interfaces to the\n given `--module`.\n --target-module-name TARGET_MODULE_NAME\n Target module name. Defaults to `${module}___gold`.\n -r, --recursive Recursively traverse module hierarchy and recreate\n hierarchy with exposed interfaces\n --dry-run Show what would be created; don't actually write to\n the filesystem.\n\nPS: Below is a temporary hack to run on the SQLalchemy output to make it work; until the `tuple`|`Tuple`|`List`|`list`|\n`name` as column-type bug is resolved:\n\n fastmod --accept-all -iF 'tuple, comment=' 'LargeBinary, comment=' ; fastmod --accept-all -iF 'tuple,\n comment=' 'LargeBinary, comment=' ; fastmod --accept-all -iF 'list, comment=' 'LargeBinary, comment=' ; fastmod --accept-all -iF 'list,\n comment=' 'LargeBinary, comment=' ; fastmod --accept-all -iF 'name, comment=' 'String, comment='\n\n---\n\n## License\n\nLicensed under either of\n\n- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or \u003chttps://apache.org/licenses/LICENSE-2.0\u003e)\n- MIT license ([LICENSE-MIT](LICENSE-MIT) or \u003chttps://opensource.org/licenses/MIT\u003e)\n\nat your option.\n\n### Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foffscale%2Fcdd-python","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foffscale%2Fcdd-python","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foffscale%2Fcdd-python/lists"}