{"id":13815107,"url":"https://github.com/openscilab/pymilo","last_synced_at":"2025-04-09T19:20:31.626Z","repository":{"id":94865458,"uuid":"576054660","full_name":"openscilab/pymilo","owner":"openscilab","description":"PyMilo: Python for ML I/O","archived":false,"fork":false,"pushed_at":"2024-04-24T08:12:53.000Z","size":761,"stargazers_count":122,"open_issues_count":21,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-04-24T09:25:54.380Z","etag":null,"topics":["artificial-intelligence","artificial-intelligence-algorithms","artificial-neural-networks","deep-learning","deserialization","devops","devops-tools","machine-learning","machine-learning-algorithms","ml","ml-models-export","ml-models-import","mlops","scikit-learn","serialization"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openscilab.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":"SUPPORTED_MODELS.md","governance":null,"roadmap":null,"authors":"AUTHORS.md","dei":null}},"created_at":"2022-12-08T22:49:33.000Z","updated_at":"2024-04-25T10:31:54.335Z","dependencies_parsed_at":"2023-09-24T07:17:31.193Z","dependency_job_id":"6e699ea5-be0e-46b9-bcf3-4652ed90a61b","html_url":"https://github.com/openscilab/pymilo","commit_stats":{"total_commits":52,"total_committers":5,"mean_commits":10.4,"dds":0.4423076923076923,"last_synced_commit":"116416d829aad7b14cc84d1e32a552e25b14373a"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscilab%2Fpymilo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscilab%2Fpymilo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscilab%2Fpymilo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openscilab%2Fpymilo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openscilab","download_url":"https://codeload.github.com/openscilab/pymilo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248094991,"owners_count":21046770,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["artificial-intelligence","artificial-intelligence-algorithms","artificial-neural-networks","deep-learning","deserialization","devops","devops-tools","machine-learning","machine-learning-algorithms","ml","ml-models-export","ml-models-import","mlops","scikit-learn","serialization"],"created_at":"2024-08-04T04:02:58.101Z","updated_at":"2025-04-09T19:20:31.596Z","avatar_url":"https://github.com/openscilab.png","language":"Python","readme":"\u003cdiv align=\"center\"\u003e\n    \u003cimg src=\"https://github.com/openscilab/pymilo/raw/main/otherfiles/logo.png\" width=\"500\" height=\"300\"\u003e\n    \u003cbr/\u003e\n    \u003cbr/\u003e\n    \u003ca href=\"https://codecov.io/gh/openscilab/pymilo\"\u003e\u003cimg src=\"https://codecov.io/gh/openscilab/pymilo/branch/main/graph/badge.svg\" alt=\"Codecov\"/\u003e\u003c/a\u003e\n    \u003ca href=\"https://badge.fury.io/py/pymilo\"\u003e\u003cimg src=\"https://badge.fury.io/py/pymilo.svg\" alt=\"PyPI version\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://anaconda.org/openscilab/pymilo\"\u003e\u003cimg src=\"https://anaconda.org/openscilab/pymilo/badges/version.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://www.python.org/\"\u003e\u003cimg src=\"https://img.shields.io/badge/built%20with-Python3-green.svg\" alt=\"built with Python3\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/openscilab/pymilo\"\u003e\u003cimg alt=\"GitHub repo size\" src=\"https://img.shields.io/github/repo-size/openscilab/pymilo\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://discord.gg/mtuMS8AjDS\"\u003e\u003cimg src=\"https://img.shields.io/discord/1064533716615049236.svg\" alt=\"Discord Channel\"\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n----------\n\n## Overview\n\u003cp align=\"justify\"\u003e\nPyMilo is an open source Python package that provides a simple, efficient, and safe way for users to export pre-trained machine learning models in a transparent way. By this, the exported model can be used in other environments, transferred across different platforms, and shared with others. PyMilo allows the users to export the models that are trained using popular Python libraries like scikit-learn, and then use them in deployment environments, or share them without exposing the underlying code or dependencies. The transparency of the exported models ensures reliability and safety for the end users, as it eliminates the risks of binary or pickle formats.\n\u003c/p\u003e\n\u003ctable\u003e\n    \u003ctr\u003e\n        \u003ctd align=\"center\"\u003ePyPI Counter\u003c/td\u003e\n        \u003ctd align=\"center\"\u003e\n\t    \u003ca href=\"https://pepy.tech/projects/pymilo\"\u003e\n\t        \u003cimg src=\"https://static.pepy.tech/badge/pymilo\" alt=\"PyPI Downloads\"\u003e\n\t    \u003c/a\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd align=\"center\"\u003eGithub Stars\u003c/td\u003e\n        \u003ctd align=\"center\"\u003e\n            \u003ca href=\"https://github.com/openscilab/pymilo\"\u003e\n                \u003cimg src=\"https://img.shields.io/github/stars/openscilab/pymilo.svg?style=social\u0026label=Stars\"\u003e\n            \u003c/a\u003e\n        \u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\u003ctable\u003e\n    \u003ctr\u003e \n        \u003ctd align=\"center\"\u003eBranch\u003c/td\u003e\n        \u003ctd align=\"center\"\u003emain\u003c/td\u003e\n        \u003ctd align=\"center\"\u003edev\u003c/td\u003e\n    \u003c/tr\u003e\n    \u003ctr\u003e\n        \u003ctd align=\"center\"\u003eCI\u003c/td\u003e\n        \u003ctd align=\"center\"\u003e\n            \u003cimg src=\"https://github.com/openscilab/pymilo/actions/workflows/test.yml/badge.svg?branch=main\"\u003e\n        \u003c/td\u003e\n        \u003ctd align=\"center\"\u003e\n            \u003cimg src=\"https://github.com/openscilab/pymilo/actions/workflows/test.yml/badge.svg?branch=dev\"\u003e\n            \u003c/td\u003e\n    \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003ctable\u003e\n\t\u003ctr\u003e \n\t\t\u003ctd align=\"center\"\u003eCode Quality\u003c/td\u003e\n\t\t\u003ctd align=\"center\"\u003e\u003ca href=\"https://www.codefactor.io/repository/github/openscilab/pymilo\"\u003e\u003cimg src=\"https://www.codefactor.io/repository/github/openscilab/pymilo/badge\" alt=\"CodeFactor\" /\u003e\u003c/a\u003e\u003c/td\u003e\n\t\t\u003ctd align=\"center\"\u003e\u003ca href=\"https://app.codacy.com/gh/openscilab/pymilo/dashboard?utm_source=gh\u0026utm_medium=referral\u0026utm_content=\u0026utm_campaign=Badge_grade\"\u003e\u003cimg src=\"https://app.codacy.com/project/badge/Grade/9eeec99ed11f4d9b86af36dc90f5f753\"\u003e\u003c/a\u003e\u003c/td\u003e\n\t\t\u003ctd align=\"center\"\u003e\u003ca href=\"https://codebeat.co/projects/github-com-openscilab-pymilo-dev\"\u003e\u003cimg alt=\"codebeat badge\" src=\"https://codebeat.co/badges/1259254f-39fc-4491-8469-17d8a43b6697\" /\u003e\u003c/a\u003e\u003c/td\u003e\n\t\u003c/tr\u003e\n\u003c/table\u003e\n\n\n## Installation\n\n### PyPI\n\n- Check [Python Packaging User Guide](https://packaging.python.org/installing/)\n- Run `pip install pymilo==1.3`\n### Source code\n- Download [Version 1.3](https://github.com/openscilab/pymilo/archive/v1.3.zip) or [Latest Source](https://github.com/openscilab/pymilo/archive/dev.zip)\n- Run `pip install .`\n\n### Conda\n\n- Check [Conda Managing Package](https://conda.io/)\n- Update Conda using `conda update conda`\n- Run `conda install -c openscilab pymilo`\n\n\n## Usage\n### Import/Export\nImagine you want to train a `LinearRegression` model representing this equation: $y = x_0 + 2x_1 + 3$. You will create data points (`X`, `y`) and train your model as follows.\n```python\nimport numpy as np\nfrom sklearn.linear_model import LinearRegression\nX = np.array([[1, 1], [1, 2], [2, 2], [2, 3]])\ny = np.dot(X, np.array([1, 2])) + 3\n# y = 1 * x_0 + 2 * x_1 + 3\nmodel = LinearRegression().fit(X, y)\npred = model.predict(np.array([[3, 5]]))\n# pred = [16.] (=1 * 3 + 2 * 5 + 3)\n```\n\nUsing PyMilo `Export` class you can easily serialize and export your trained model into a JSON file.\n```python\nfrom pymilo import Export\nExport(model).save(\"model.json\")\n```\n\nYou can check out your model as a JSON file now.\n```json\n{\n    \"data\": {\n        \"fit_intercept\": true,\n        \"copy_X\": true,\n        \"n_jobs\": null,\n        \"positive\": false,\n        \"n_features_in_\": 2,\n        \"coef_\": {\n            \"pymiloed-ndarray-list\": [\n                1.0000000000000002,\n                1.9999999999999991\n            ],\n            \"pymiloed-ndarray-dtype\": \"float64\",\n            \"pymiloed-ndarray-shape\": [\n                2\n            ],\n            \"pymiloed-data-structure\": \"numpy.ndarray\"\n        },\n        \"rank_\": 2,\n        \"singular_\": {\n            \"pymiloed-ndarray-list\": [\n                1.618033988749895,\n                0.6180339887498948\n            ],\n            \"pymiloed-ndarray-dtype\": \"float64\",\n            \"pymiloed-ndarray-shape\": [\n                2\n            ],\n            \"pymiloed-data-structure\": \"numpy.ndarray\"\n        },\n        \"intercept_\": {\n            \"value\": 3.0000000000000018,\n            \"np-type\": \"numpy.float64\"\n        }\n    },\n    \"sklearn_version\": \"1.4.2\",\n    \"pymilo_version\": \"0.8\",\n    \"model_type\": \"LinearRegression\"\n}\n```\nYou can see all the learned parameters of the model in this file and change them if you want. This JSON representation is a transparent version of your model.\n\nNow let's load it back. You can do it easily by using PyMilo `Import` class.\n```python\nfrom pymilo import Import\nmodel = Import(\"model.json\").to_model()\npred = model.predict(np.array([[3, 5]]))\n# pred = [16.] (=1 * 3 + 2 * 5 + 3)\n```\nThis loaded model is exactly the same as the original trained model.\n\n### ML streaming\nYou can easily serve your ML model from a remote server using `ML streaming` feature of PyMilo.\n\n⚠️ `ML streaming` feature exists in versions `\u003e=1.0`\n\n⚠️ In order to use `ML streaming` feature, make sure you've installed the `streaming` mode of PyMilo\n\nYou can choose either `REST` or `WebSocket` as the communication medium protocol.\n\n#### Server\nLet's assume you are in the remote server and you want to import the exported JSON file and start serving your model through `REST` protocol!\n```python\nfrom pymilo import Import\nfrom pymilo.streaming import PymiloServer, CommunicationProtocol\nmy_model = Import(\"model.json\").to_model()\ncommunicator = PymiloServer(\n    model=my_model,\n    port=8000,\n    communication_protocol=CommunicationProtocol[\"REST\"],\n    ).communicator\ncommunicator.run()\n```\nNow `PymiloServer` runs on port `8000` and exposes REST API to `upload`, `download` and retrieve **attributes** either **data attributes** like `model._coef` or **method attributes** like `model.predict(x_test)`.\n\n#### Client\nBy using `PymiloClient` you can easily connect to the remote `PymiloServer` and execute any functionalities that the given ML model has, let's say you want to run `predict` function on your remote ML model and get the result:\n```python\nfrom pymilo.streaming import PymiloClient, CommunicationProtocol\npymilo_client = PymiloClient(\n    mode=PymiloClient.Mode.LOCAL,\n    server_url=\"SERVER_URL\",\n    communication_protocol=CommunicationProtocol[\"REST\"],\n    )\npymilo_client.toggle_mode(PymiloClient.Mode.DELEGATE)\nresult = pymilo_client.predict(x_test)\n```\n\nℹ️ If you've deployed `PymiloServer` locally (on port `8000` for instance), then `SERVER_URL` would be `http://127.0.0.1:8000` or `ws://127.0.0.1:8000` based on the selected protocol for the communication medium.\n\nYou can also download the remote ML model into your local and execute functions locally on your model.\n\nCalling `download` function on `PymiloClient` will sync the local model that `PymiloClient` wraps upon with the remote ML model, and it doesn't save model directly to a file.\n\n```python\npymilo_client.download()\n```\nIf you want to save the ML model to a file in your local, you can use `Export` class.\n```python\nfrom pymilo import Export\nExport(pymilo_client.model).save(\"model.json\")\n```\nNow that you've synced the remote model with your local model, you can run functions.\n```python\npymilo_client.toggle_mode(mode=PymiloClient.Mode.LOCAL)\nresult = pymilo_client.predict(x_test)\n```\n`PymiloClient` wraps around the ML model, either to the local ML model or the remote ML model, and you can work with `PymiloClient` in the exact same way that you did with the ML model, you can run exact same functions with same signature.\n\nℹ️ Through the usage of `toggle_mode` function you can specify whether `PymiloClient` applies requests on the local ML model `pymilo_client.toggle_mode(mode=Mode.LOCAL)` or delegates it to the remote server `pymilo_client.toggle_mode(mode=Mode.DELEGATE)`\n\n\n## Supported ML models\n| scikit-learn | PyTorch | \n| ---------------- | ---------------- | \n| Linear Models \u0026#x2705; | - | \n| Neural Networks \u0026#x2705; | -  | \n| Trees \u0026#x2705; | -  | \n| Clustering \u0026#x2705; | -  | \n| Naïve Bayes \u0026#x2705; | -  | \n| Support Vector Machines (SVMs) \u0026#x2705; | -  | \n| Nearest Neighbors \u0026#x2705; | -  |  \n| Ensemble Models \u0026#x2705; | - | \n| Pipeline Model \u0026#x2705; | - |\n| Preprocessing Models \u0026#x2705; | - |\n| Cross Decomposition Models \u0026#x2705; | - |\n| Feature Extractor Models \u0026#x2705; | - |\n\n\nDetails are available in [Supported Models](https://github.com/openscilab/pymilo/blob/main/SUPPORTED_MODELS.md).\n\n## Issues \u0026 bug reports\n\nJust fill an issue and describe it. We'll check it ASAP! or send an email to [pymilo@openscilab.com](mailto:pymilo@openscilab.com \"pymilo@openscilab.com\"). \n\n- Please complete the issue template\n \nYou can also join our discord server\n\n\u003ca href=\"https://discord.gg/mtuMS8AjDS\"\u003e\n  \u003cimg src=\"https://img.shields.io/discord/1064533716615049236.svg?style=for-the-badge\" alt=\"Discord Channel\"\u003e\n\u003c/a\u003e\n\n## Acknowledgments\n\n[Python Software Foundation (PSF)](https://www.python.org/psf/) grants PyMilo library partially for versions **1.0, 1.1**. [PSF](https://www.python.org/psf/) is the organization behind Python. Their mission is to promote, protect, and advance the Python programming language and to support and facilitate the growth of a diverse and international community of Python programmers.\n\n\u003ca href=\"https://www.python.org/psf/\"\u003e\u003cimg src=\"https://github.com/openscilab/pymilo/raw/main/otherfiles/psf.png\" height=\"65px\" alt=\"Python Software Foundation\"\u003e\u003c/a\u003e\n\n[Trelis Research](https://trelis.com/) grants PyMilo library partially for version **1.0**. [Trelis Research](https://trelis.com/) provides tools and tutorials for businesses and developers looking to fine-tune and deploy large language models.\n\n\u003ca href=\"https://trelis.com/\"\u003e\u003cimg src=\"https://trelis.com/wp-content/uploads/2023/10/android-chrome-512x512-1.png\" height=\"75px\" alt=\"Trelis Research\"\u003e\u003c/a\u003e\n\n## Cite\n\nIf you use PyMilo in your research, we would appreciate citations to the following paper :\n\n[Rostami, A., Haghighi, S., Sabouri, S., \u0026 Zolanvari, A. (2024). *PyMilo: A Python Library for ML I/O*. *arXiv e-prints*, arXiv-2501.](https://arxiv.org/abs/2501.00528)\n\n\u003ctable\u003e\n\t\u003ctr\u003e\n\t\t\u003ctd align=\"center\"\u003eZenodo\u003c/td\u003e\n\t\t\u003ctd align=\"center\"\u003e\u003ca href=\"https://doi.org/10.5281/zenodo.14713078\"\u003e\u003cimg src=\"https://zenodo.org/badge/DOI/10.5281/zenodo.14713078.svg\" alt=\"DOI 10.5281/zenodo.14713078\"\u003e\u003c/a\u003e\u003c/td\u003e\n\t\u003c/tr\u003e\n\u003c/table\u003e\n\n```bibtex\n@article{rostami2024pymilo,\n  title={PyMilo: A Python Library for ML I/O},\n  author={Rostami, AmirHosein and Haghighi, Sepand and Sabouri, Sadra and Zolanvari, Alireza},\n  journal={arXiv e-prints},\n  pages={arXiv--2501},\n  year={2024}\n}\n```\n\n## Show your support\n\n\n### Star this repo\n\nGive a ⭐️ if this project helped you!\n\n### Donate to our project\nIf you do like our project and we hope that you do, can you please support us? Our project is not and is never going to be working for profit. We need the money just so we can continue doing what we do ;-) .\t\t\t\n\n\u003ca href=\"https://openscilab.com/#donation\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/openscilab/pymilo/raw/main/otherfiles/donation.png\" height=\"90px\" width=\"270px\" alt=\"PyMilo Donation\"\u003e\u003c/a\u003e\n","funding_links":[],"categories":["Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenscilab%2Fpymilo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenscilab%2Fpymilo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenscilab%2Fpymilo/lists"}