https://github.com/samuelmarks/ml-params-tensorflow
TensorFlow implementation of ml-params CLI API
https://github.com/samuelmarks/ml-params-tensorflow
Last synced: 3 months ago
JSON representation
TensorFlow implementation of ml-params CLI API
- Host: GitHub
- URL: https://github.com/samuelmarks/ml-params-tensorflow
- Owner: SamuelMarks
- License: apache-2.0
- Created: 2020-07-02T06:46:32.000Z (about 6 years ago)
- Default Branch: master
- Last Pushed: 2021-04-08T11:03:29.000Z (about 5 years ago)
- Last Synced: 2025-03-23T01:28:01.804Z (over 1 year ago)
- Language: Python
- Homepage:
- Size: 480 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
Awesome Lists containing this project
README
ml_params_tensorflow
===============
[](https://opensource.org/licenses/Apache-2.0)
[](https://github.com/SamuelMarks/ml-params-tensorflow/actions)
[](https://codecov.io/gh/SamuelMarks/ml-params-tensorflow)
[](https://github.com/psf/black)
[](https://pycqa.github.io/isort/)
[TensorFlow](https://tensorflow.org) implementation of the [ml-params](https://github.com/SamuelMarks/ml-params) API and CLI.
The purpose of ml-params is to expose the hooks and levers of ML experiments for external usage, e.g., in GUIs, CLIs, REST & RPC APIs, and parameter and hyperparameter optimisers.
## Install dependencies
pip install -r requirements.txt
## Install package
pip install .
## Usage
After installing as above, follow usage from [ml-params](https://github.com/SamuelMarks/ml-params)
## Sibling projects
| Google | Other vendors |
| -------| ------------- |
| _[tensorflow](https://github.com/SamuelMarks/ml-params-tensorflow)_ | [pytorch](https://github.com/SamuelMarks/ml-params-pytorch) |
| [keras](https://github.com/SamuelMarks/ml-params-keras) | [skorch](https://github.com/SamuelMarks/ml-params-skorch) |
| [flax](https://github.com/SamuelMarks/ml-params-flax) | [sklearn](https://github.com/SamuelMarks/ml-params-sklearn) |
| [trax](https://github.com/SamuelMarks/ml-params-trax) | [xgboost](https://github.com/SamuelMarks/ml-params-xgboost) |
| [jax](https://github.com/SamuelMarks/ml-params-jax) | [cntk](https://github.com/SamuelMarks/ml-params-cntk) |
## Related official projects
- [ml-prepare](https://github.com/SamuelMarks/ml-prepare)
## Development guide
To make the development of _ml-params-tensorflow_ type safer and maintain consistency with the other ml-params implementing projects, the [cdd](https://github.com/offscale/cdd-python) was created.
When TensorFlow itself changes—i.e., a new major version of TensorFlow is releases—then run the `sync_properties`, as shown in the module-level docstring here [`ml_params_tensorflow/ml_params/type_generators.py`](ml_params_tensorflow/ml_params/type_generators.py);
To synchronise all the various other APIs, edit one and it'll translate to the others, but make sure you select which one is the gold-standard.
### Dependencies
$ pip install python-cdd autoflake black isort
[`fd`](https://github.com/sharkdp/fd#installation) (or you can just use `find`, rewrite to `find -type f -name '*.py' -exec`)
### Examples
NOTE: The below steps are available in a script within [sync_scripts](sync_scripts).
As an example, using the `class TensorFlowTrainer` methods as truth, this will update the CLI parsers and config classes:
$ python -m cdd sync --class 'ml_params_tensorflow/ml_params/config.py' \
--class-name 'TrainConfig' \
--function 'ml_params_tensorflow/ml_params/trainer.py' \
--function-name 'TensorFlowTrainer.train' \
--argparse-function 'ml_params_tensorflow/ml_params/cli.py' \
--argparse-function-name 'train_parser' \
--truth 'function'
$ python -m cdd sync --class 'ml_params_tensorflow/ml_params/config.py' \
--class-name 'LoadDataConfig' \
--function 'ml_params_tensorflow/ml_params/trainer.py' \
--function-name 'TensorFlowTrainer.load_data' \
--argparse-function 'ml_params_tensorflow/ml_params/cli.py' \
--argparse-function-name 'load_data_parser' \
--truth 'function'
$ python -m cdd sync --class 'ml_params_tensorflow/ml_params/config.py' \
--class-name 'LoadModelConfig' \
--function 'ml_params_tensorflow/ml_params/trainer.py' \
--function-name 'TensorFlowTrainer.load_model' \
--argparse-function 'ml_params_tensorflow/ml_params/cli.py' \
--argparse-function-name 'load_model_parser' \
--truth 'function'
Another example, that you'd run before ^, to generate custom config CLI parsers for members of `tf.keras.losses`:
$ python -m cdd gen --name-tpl '{name}Config' \
--input-mapping 'ml_params_tensorflow.ml_params.type_generators.exposed_losses' \
--prepend '""" Generated Loss config classes """\nimport tensorflow as tf\n' \
--imports-from-file 'tf.keras.losses.Loss' \
--type 'argparse' \
--output-filename 'ml_params_tensorflow/ml_params/losses.py'
There's a bit of boilerplate here, so let's automate it:
$ for name in 'callbacks' 'losses' 'metrics' 'optimizers'; do
rm 'ml_params_tensorflow/ml_params/'"$name"'.py';
python -m ml_params_tensorflow.ml_params.cdd_cli_gen "$name" 2>/dev/null | xargs python -m cdd gen;
done
Cleanup the code everywhere, removing unused imports and autolinting/autoformatting:
$ fd -HIepy -x sh -c 'autoflake --remove-all-unused-imports -i "$0" && isort --atomic "$0" && python -m black "$0"' {} \;
---
## License
Licensed under either of
- Apache License, Version 2.0 ([LICENSE-APACHE](LICENSE-APACHE) or )
- MIT license ([LICENSE-MIT](LICENSE-MIT) or )
at your option.
### Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.