Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/google/ml-metadata
For recording and retrieving metadata associated with ML developer and data scientist workflows.
https://github.com/google/ml-metadata
Last synced: 9 days ago
JSON representation
For recording and retrieving metadata associated with ML developer and data scientist workflows.
- Host: GitHub
- URL: https://github.com/google/ml-metadata
- Owner: google
- License: apache-2.0
- Created: 2019-01-15T21:02:09.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-04-26T20:02:09.000Z (7 months ago)
- Last Synced: 2024-05-18T17:45:04.731Z (6 months ago)
- Language: C++
- Homepage: https://www.tensorflow.org/tfx/guide/mlmd
- Size: 2.82 MB
- Stars: 601
- Watchers: 29
- Forks: 134
- Open Issues: 40
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-python-machine-learning-resources - GitHub - 26% open · ⏱️ 23.08.2022): (工作流程和实验跟踪)
- awesome-production-machine-learning - ML Metadata - metadata.svg?style=social) - a library for recording and retrieving metadata associated with ML developer and data scientist workflows. (Metadata Management)
README
# ML Metadata
[](https://github.com/google/ml-metadata)
[](https://badge.fury.io/py/ml-metadata)
*ML Metadata (MLMD)* is a library for recording and retrieving metadata
associated with ML developer and data scientist workflows.
NOTE: ML Metadata may be backwards incompatible before version 1.0.
## Getting Started
For more background on MLMD and instructions on using it, see the
[getting started guide](https://github.com/google/ml-metadata/blob/master/g3doc/get_started.md)
## Installing from PyPI
The recommended way to install ML Metadata is to use the
[PyPI package](https://pypi.org/project/ml-metadata/):
```bash
pip install ml-metadata
```
Then import the relevant packages:
```python
from ml_metadata import metadata_store
from ml_metadata.proto import metadata_store_pb2
```
### Nightly Packages
ML Metadata (MLMD) also hosts nightly packages at
https://pypi-nightly.tensorflow.org on Google Cloud. To install the latest
nightly package, please use the following command:
```bash
pip install --extra-index-url https://pypi-nightly.tensorflow.org/simple ml-metadata
```
## Installing with Docker
This is the recommended way to build ML Metadata under Linux, and is
continuously tested at Google.
Please first install `docker` and `docker-compose` by following the directions:
[docker](https://docs.docker.com/install/);
[docker-compose](https://docs.docker.com/compose/install/).
Then, run the following at the project root:
```bash
DOCKER_SERVICE=manylinux-python${PY_VERSION}
sudo docker-compose build ${DOCKER_SERVICE}
sudo docker-compose run ${DOCKER_SERVICE}
```
where `PY_VERSION` is one of `{39, 310, 311}`.
A wheel will be produced under `dist/`, and installed as follows:
```shell
pip install dist/*.whl
```
## Installing from source
### 1. Prerequisites
To compile and use ML Metadata, you need to set up some prerequisites.
#### Install Bazel
If Bazel is not installed on your system, install it now by following [these
directions](https://bazel.build/versions/master/docs/install.html).
#### Install cmake
If cmake is not installed on your system, install it now by following [these
directions](https://cmake.org/install/).
### 2. Clone ML Metadata repository
```shell
git clone https://github.com/google/ml-metadata
cd ml-metadata
```
Note that these instructions will install the latest master branch of ML
Metadata. If you want to install a specific branch (such as a release branch),
pass `-b ` to the `git clone` command.
### 3. Build the pip package
ML Metadata uses Bazel to build the pip package from source:
```shell
python setup.py bdist_wheel
```
You can find the generated `.whl` file in the `dist` subdirectory.
### 4. Install the pip package
```shell
pip install dist/*.whl
```
### 5.(Optional) Build the grpc server
ML Metadata uses Bazel to build the c++ binary from source:
```shell
bazel build -c opt --define grpc_no_ares=true //ml_metadata/metadata_store:metadata_store_server
```
## Supported platforms
MLMD is built and tested on the following 64-bit operating systems:
* macOS 10.14.6 (Mojave) or later.
* Ubuntu 20.04 or later.
* [DEPRECATED] Windows 10 or later. For a Windows-compatible library, please
refer to MLMD 1.14.0 or earlier versions.