{"id":13639956,"url":"https://github.com/operatorai/modelstore","last_synced_at":"2025-05-14T21:08:13.670Z","repository":{"id":37010070,"uuid":"297089520","full_name":"operatorai/modelstore","owner":"operatorai","description":"🏬 modelstore is a Python library that allows you to version, export, and save a machine learning model to your filesystem or a cloud storage provider.","archived":false,"fork":false,"pushed_at":"2025-01-14T20:37:51.000Z","size":767,"stargazers_count":387,"open_issues_count":13,"forks_count":25,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-30T20:01:43.591Z","etag":null,"topics":["data-science","keras","machine-learning","mlops","modelstore","python-library","pytorch","s3-storage","scikit-learn","tensorflow","transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/operatorai.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-09-20T14:06:18.000Z","updated_at":"2025-03-26T19:59:54.000Z","dependencies_parsed_at":"2023-11-09T12:29:54.843Z","dependency_job_id":"0f42ee19-3bda-4c60-b5b6-16baeb3d0ef6","html_url":"https://github.com/operatorai/modelstore","commit_stats":{"total_commits":274,"total_committers":9,"mean_commits":"30.444444444444443","dds":"0.10948905109489049","last_synced_commit":"00da4e1b26883947d1754502f20c796cfc33832f"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/operatorai%2Fmodelstore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/operatorai%2Fmodelstore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/operatorai%2Fmodelstore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/operatorai%2Fmodelstore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/operatorai","download_url":"https://codeload.github.com/operatorai/modelstore/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247550672,"owners_count":20956985,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-science","keras","machine-learning","mlops","modelstore","python-library","pytorch","s3-storage","scikit-learn","tensorflow","transformer"],"created_at":"2024-08-02T01:01:06.473Z","updated_at":"2025-04-06T21:03:39.206Z","avatar_url":"https://github.com/operatorai.png","language":"Python","readme":"# modelstore\n\n`modelstore` is a Python library that allows you to version, export, save and download machine learning models in your choice of storage.\n\n[![Downloads](https://pepy.tech/badge/modelstore)](https://pepy.tech/project/modelstore) [![Downloads](https://pepy.tech/badge/modelstore/month)](https://pepy.tech/project/modelstore)\n\n💭  Give us feedback by completing this survey: https://forms.gle/XShU3zrZcnLRWsk36\n\n[![](https://img.shields.io/static/v1?label=Sponsor\u0026message=%E2%9D%A4\u0026logo=GitHub\u0026color=%23fe8e86)](https://github.com/sponsors/nlathia/)\n\n## modelstore is an open source model registry\n\n✅ No tracking server required\n* Store models on a local file system or in a bucket\n* Support for multiple clouds (AWS, GCP, Azure)\n\n✅ Upload and version all your models\n* Models are versioned on each upload\n* Replaces all the boiler plate code you need to save models\n\n✅ Manage models by domains and states\n* List models in a domain\n* Create model states and manage which state a model is in\n\n✅ Download or load straight into memory\n* Download models by id\n* Load models straight from your storage back into memory\n\n✅ Use as a command line tool\n* Download models from the command line\n\nFor more details, please refer to [the documentation](https://modelstore.readthedocs.io/en/latest/).\n\n## modelstore is being built in the open\n\n💬 Come and find us in the [MLOps Community Slack](https://go.mlops.community/slack)'s `#oss-modelstore` channel.\n\n## Installation\n\n```python\npip install modelstore\n```\n\n## Supported storage types\n\n* AWS S3 Bucket ([example](https://github.com/operatorai/modelstore/blob/b096275018674243835d21102f75b6270dfa2c97/examples/examples-by-storage/modelstores.py#L17-L21))\n* Azure Blob Storage ([example](https://github.com/operatorai/modelstore/blob/b096275018674243835d21102f75b6270dfa2c97/examples/examples-by-storage/modelstores.py#L24-L31))\n* Google Cloud Storage Bucket ([example](https://github.com/operatorai/modelstore/blob/b096275018674243835d21102f75b6270dfa2c97/examples/examples-by-storage/modelstores.py#L34-L41))\n* Any s3-compatible object storage that you can access via [MinIO](https://min.io/)\n* A filesystem directory ([example](https://github.com/operatorai/modelstore/blob/b096275018674243835d21102f75b6270dfa2c97/examples/examples-by-storage/modelstores.py#L44-L49))\n\n\n## Supported machine learning libraries\n\n* [Annoy](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/annoy_example.py)\n* [Catboost](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/catboost_example.py)\n* [CausalML](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/causalml_example.py)\n* [Fast.AI](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/fastai_example.py)\n* [Gensim](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/gensim_example.py)\n* [Keras](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/keras_example.py)\n* [LightGBM](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/lightgbm_example.py)\n* [Mxnet](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/mxnet_example.py)\n* [Onnx](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/onnx_sklearn_example.py)\n* [Prophet](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/prophet_example.py)\n* [PyTorch](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/pytorch_example.py)\n* [PyTorch Lightning](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/pytorch_lightning_example.py)\n* [Scikit Learn](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/sklearn_example.py)\n* [Skorch](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/skorch_example.py)\n* [Shap](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/shap_example.py)\n* [Spark ML Lib](https://spark.apache.org/)\n* [Tensorflow](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/tensorflow_example.py)\n* Transformers - there are several examples in [this directory](https://github.com/operatorai/modelstore/tree/main/examples/examples-by-ml-library/libraries/huggingface)\n* [XGBoost](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/xgboost_example.py)\n\nIs there a machine learning framework that is missing? \n* Save your model and then upload it [as a raw file](https://github.com/operatorai/modelstore/blob/main/examples/examples-by-ml-library/libraries/raw_file_example.py).\n* Feel free to [open an issue](https://github.com/operatorai/modelstore/issues)\n\n## Read more about modelstore\n\n* [35 Hidden Python Libraries That Are Absolute Gems](https://www.blog.dailydoseofds.com/p/35-gem-py-libs), March 2023\n* [Evidently.AI AMA with Neal Lathia](https://www.evidentlyai.com/blog/ama-neal-lathia), January 2023\n* [MLOps Model Stores: Definition, Functionality, Tools Review](https://neptune.ai/blog/mlops-model-stores), January 2023\n* [Monzo's machine learning stack](https://monzo.com/blog/2022/04/26/monzos-machine-learning-stack), April 2022\n* [Data Talks Club Minis: Model Store](https://www.youtube.com/watch?v=85BWnKmOZl8), July 2021\n* [Model arterfacts: the war stories](https://nlathia.github.io/2020/09/Model-artifacts-war-stories.html), September 2020\n\n## Example Usage\n\n### Colab Notebook\n\nThere is a [full example in this Colab notebook](https://colab.research.google.com/drive/1yEY6wy68k7TlHzm8iJMKKBG_Pl-MGZUe?usp=sharing).\n\n### Python Script\n\n```python\nfrom modelstore import ModelStore\n# And your other imports\n\n# Train your model\nclf = RandomForestClassifier(n_estimators=10)\nclf = clf.fit(X, Y)\n\n# Create a model store that uses a one of the storage options\n# In this example, the model store is created with a GCP bucket\nmodel_store = ModelStore.from_gcloud(\n   project_name=\"my-project\",\n   bucket_name=\"my-bucket\",\n)\n\n# Upload the archive to your model store\ndomain = \"example-model\"\nmeta_data = model_store.upload(domain, model=clf)\n\n# Print the meta-data about the model\nprint(json.dumps(meta_data, indent=4))\n\n# Load the model back!\nclf = model_store.load(domain=model_domain, model_id=meta[\"model\"][\"model_id\"])\n```\n\n## License\n\nCopyright 2020 Neal Lathia\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","funding_links":["https://github.com/sponsors/nlathia/"],"categories":["Model and Data Versioning","Model, Data and Experiment Tracking","Python"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foperatorai%2Fmodelstore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foperatorai%2Fmodelstore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foperatorai%2Fmodelstore/lists"}