https://github.com/amine-akrout/tensorflow-tifecycle-management-with-mlflow
Creating a Dockerized stack environment using MLflow, mysql and Minio to manage the lifecycle of TensorFlow models.
https://github.com/amine-akrout/tensorflow-tifecycle-management-with-mlflow
bert cnn deep-learning lstm machine-learning minio mlflow mlops mysql nlp python s3 swivel tensorboard tensorflow tf-serving
Last synced: about 2 months ago
JSON representation
Creating a Dockerized stack environment using MLflow, mysql and Minio to manage the lifecycle of TensorFlow models.
- Host: GitHub
- URL: https://github.com/amine-akrout/tensorflow-tifecycle-management-with-mlflow
- Owner: amine-akrout
- Created: 2021-10-19T13:41:27.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-03-15T13:32:30.000Z (about 1 year ago)
- Last Synced: 2025-03-24T06:36:25.144Z (2 months ago)
- Topics: bert, cnn, deep-learning, lstm, machine-learning, minio, mlflow, mlops, mysql, nlp, python, s3, swivel, tensorboard, tensorflow, tf-serving
- Language: Python
- Homepage:
- Size: 10.4 MB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Tensorflow models LifeCycle managemenet [](https://github.com/amine-akrout/Tensorflow-Lifecycle-management-with-MLFlow/actions)
This repository contains a docker-compose stack with MLflow, mysql, phpmyadmin and Minio. The networking is set up so running containers could communicate.
## Quickstart
The easiest way to understand the setup is by diving into it and interacting with it.
### 1. Clone the repository and create a virtual environment
```
git clone https://github.com/amine-akrout/Tensorflow-Lifecycle-management-with-MLFlow.git
cd Tensorflow-Lifecycle-management-with-MLFlow
```
This will clone the repo
```
pip install virtualenv
```
if you don't already have virtualenv installed
virtualenv venv to create your new environment (called 'venv' here)
```
source venv/bin/activate
```
to enter the virtual environment
```
pip install -r requirements.txt
```
to install the requirements in the current environment
### 2. Lunch the Docker Stack
make sure you have Docker installed
```
docker-compose build
```
this will build mlflow_server image
```
docker-compose up -d
```
after this, you have all container running so you can start training ML models
#### Detail Summary
| **Container** | **Port** |
|--------------- |---------- |
| MLflow_server | 5000 |
| Minio | 9000 |
| Mysql | 80 |
| phpmyadmin | 3306 |
### 3. Start Training the ML models
```
python .\TensorFlow_training\baseline_training.py
```
```
python .\TensorFlow_training\LSTM_training.py
```
```
python .\TensorFlow_training\CNN_training.py
```
```
python .\TensorFlow_training\swivel_training.py
```
```
python .\TensorFlow_training\BERT_training.py
```
### 4. Visualize and compare models performances with MLflow UI to pick the best one
Access the MLflow Dashboard: http://localhost:5000
you can also access and query the database : http://localhost:3306
the model artifacts are saved in minio, to access : http://localhost:9000
And finally to visualize tensorboard charts run the following command :
```
tensorboard --logdir .\TensorFlow_training\logs\fit
```
the open it in the browser http://localhost:6000
MLflow UI
Mysql database in phpmyadmin
Models artifacts in Minio
tensorboard visualization
## Web app demo
demo