https://github.com/axent-pl/ax-ml-ops

axent.pl MLOps application stack for running your ML experiments locally
https://github.com/axent-pl/ax-ml-ops

airflow docker docker-compose jupyter-notebook kaggle mlflow mlops optuna python rabbitmq

Last synced: 15 days ago
JSON representation

axent.pl MLOps application stack for running your ML experiments locally

• Host: GitHub
• URL: https://github.com/axent-pl/ax-ml-ops
• Owner: axent-pl
• License: gpl-3.0
• Created: 2023-01-13T07:24:39.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2023-04-04T13:07:30.000Z (almost 2 years ago)
• Last Synced: 2024-11-09T07:24:45.076Z (2 months ago)
• Topics: airflow, docker, docker-compose, jupyter-notebook, kaggle, mlflow, mlops, optuna, python, rabbitmq
• Language: Python
• Homepage:
• Size: 2.18 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# MLOps application stack

This project shows how you may integrate some of the popular tools to perform Data Science experiments.

## Components

* **S3 (Minio)** exposed at http://localhost:9000

* **Airflow** exposed at http://localhost:8000

* **Jupyter notebook** exposed at http://localhost:8800

* **MLFlow** exposed at http://localhost:5000

* **Optuna dashboard** exposed at http://localhost:8080

* **PostgreSQL**

* **RabbitMQ**

## Usage scenario

1. Start **ax-ml-ops** application stack with

    ```shell

    ./stack-up.sh

    ```

2. Enable following pipelines in **Airflow**

    * kaggle-spaceship-titanic-data

    * kaggle-spaceship-titanic-model

3. Execute *kaggle-spaceship-titanic-data* pipeline with **Airflow**

4. Monitor key metrics with **MLFlow**

5. Monitor model hyperparameter tuning with **Optuna dashboard**

6. Stop **ax-ml-ops** application stack and remove all its assets (without images) with

    ```shell

    ./stack-down.sh

    ```