{"id":16383721,"url":"https://github.com/kartik4949/autodeploy","last_synced_at":"2025-03-21T02:30:56.529Z","repository":{"id":46676416,"uuid":"394000201","full_name":"kartik4949/AutoDeploy","owner":"kartik4949","description":"AutoDeploy is a single configuration deployment library","archived":false,"fork":false,"pushed_at":"2021-09-30T17:41:48.000Z","size":36094,"stargazers_count":41,"open_issues_count":7,"forks_count":12,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-17T19:52:08.261Z","etag":null,"topics":["deployment","deployment-automation","monioring","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kartik4949.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-08T15:35:50.000Z","updated_at":"2023-07-12T19:58:20.000Z","dependencies_parsed_at":"2022-09-03T21:12:41.272Z","dependency_job_id":null,"html_url":"https://github.com/kartik4949/AutoDeploy","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kartik4949%2FAutoDeploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kartik4949%2FAutoDeploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kartik4949%2FAutoDeploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kartik4949%2FAutoDeploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kartik4949","download_url":"https://codeload.github.com/kartik4949/AutoDeploy/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244725452,"owners_count":20499600,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deployment","deployment-automation","monioring","tensorflow"],"created_at":"2024-10-11T04:09:31.347Z","updated_at":"2025-03-21T02:30:56.084Z","avatar_url":"https://github.com/kartik4949.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eAutoDeploy\n\u003ca href=\"https://www.producthunt.com/posts/autodeploy?utm_source=badge-featured\u0026utm_medium=badge\u0026utm_souce=badge-autodeploy\" target=\"_blank\"\u003e\u003cimg src=\"https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=311994\u0026theme=light\" alt=\"AutoDeploy - Automated machine learning model deployment. | Product Hunt\" style=\"width: 250px; height: 54px;\" width=\"250\" height=\"54\" /\u003e\u003c/a\u003e \u003c/h1\u003e\n\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"https://cdn.rawgit.com/sindresorhus/awesome/d7305f38d29fed78fa85652e3a63e154dd8e8829/media/badge.svg\" alt=\"Awesome Badge\"/\u003e\n\n\u003cimg src=\"https://img.shields.io/static/v1?label=%F0%9F%8C%9F\u0026message=If%20Useful\u0026style=style=flat\u0026color=BC4E99\" alt=\"Star Badge\"/\u003e\n\u003cbr\u003e\n\u003ca href=\"https://github.com/abhisheknaiidu/awesome-github-profile-readme/stargazers\"\u003e\u003cimg src=\"https://img.shields.io/github/stars/kartik4949/AutoDeploy\" alt=\"Stars Badge\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/abhisheknaiidu/awesome-github-profile-readme/network/members\"\u003e\u003cimg src=\"https://img.shields.io/github/forks/kartik4949/AutoDeploy\" alt=\"Forks Badge\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/abhisheknaiidu/awesome-github-profile-readme/pulls\"\u003e\u003cimg src=\"https://img.shields.io/github/issues-pr/kartik4949/AutoDeploy\" alt=\"Pull Requests Badge\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/abhisheknaiidu/awesome-github-profile-readme/issues\"\u003e\u003cimg src=\"https://img.shields.io/github/issues/kartik4949/AutoDeploy\" alt=\"Issues Badge\"/\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/kartik4949/AutoDeploy/graphs/contributors\"\u003e\u003cimg alt=\"GitHub contributors\" src=\"https://img.shields.io/github/contributors/kartik4949/AutoDeploy?color=2b9348\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/kartik4949/AutoDeploy/blob/master/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/kartik4949/AutoDeploy?color=2b9348\" alt=\"License Badge\"/\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n## What is AutoDeploy?\u003cimg src=\"https://c.tenor.com/jfj-uQd1eLUAAAAi/fire-space.gif\" width=\"40\" height=\"40\" /\u003e\n\n\u003cimg src=\"https://github.com/kartik4949/AutoDeploy/blob/dev/assests/src.png\" alt=\"Autodeploy src\"/\u003e\nA one liner : \nFor the DevOps nerds, AutoDeploy allows configuration based MLOps.\n\nFor the rest :\nSo you're a data scientist and have the greatest model on planet earth to classify dogs and cats! :). What next? It's a steeplearning cusrve from building your model to getting it to production. MLOps, Docker, Kubernetes, asynchronous, prometheus, logging, monitoring, versioning etc. Much more to do right before you  The immediate next thoughts and tasks are\n\n- How do you get it out to your consumer to use as a service.  \n- How do you monitor its use?\n- How do you test your model once deployed? And it can get trickier once you have multiple versions of your model. How do you perform\nA/B testing?\n- Can i configure custom metrics and monitor them?\n- What if my data distribution changes in production - how can i monitor data drift?\n- My models use different frameworks. Am i covered?\n... and many more.\n\n# Architecture\n\u003cimg src=\"https://github.com/kartik4949/AutoDeploy/blob/main/assests/AutoDeploy_architecture.png\" alt=\"AutoDeploy architecture\"/\u003e\n\n\nWhat if you could only configure a single file and get up and running with a single command. **That is what AutoDeploy is!**\n\nRead our [documentation](https://github.com/kartik4949/AutoDeploy/wiki) to know how to get setup and get to serving your models.\n\n# AutoDeploy monitoring dashboard\n\u003cimg src=\"https://miro.medium.com/max/875/1*9lIpenjWAmGydJBhD_zFsg.png\" alt=\"AutoDeploy dashboard\"/\u003e\n\n\u003cimg src=\"https://miro.medium.com/max/875/1*3jsctf-uWHDYMJSXy04nRQ.png\" alt=\"AutoDeploy dashboard\"/\u003e\n\nand many more...\n\n# Feature Support.\n\n- [x] Single Configuration file support.\n- [x] Enterprise deployment architecture.\n- [x] Logging.\n- [x] Grafana Dashboards.\n- [x] Dynamic Database.\n- [x] Data Drift Monitoring.\n- [x] Async Model Monitoring.\n- [x] Network traffic monitoring.\n- [x] Realtime traffic simulation.\n- [x] Autoscaling of services. \n- [x] Kubernetes.\n- [x] Preprocess configuration.\n- [x] Posprocess configuration.\n- [x] Custom metrics configuration. \n\n## Prerequisites\n- Install docker \n  - For Ubuntu (and Linux distros) - [Install Docker on Ubuntu](https://docs.docker.com/engine/install/ubuntu/#installation-methods)  \n  - For Windows - [Install Docker on Windows](https://docs.docker.com/desktop/windows/install/)\n  - For Mac - \n\n- Install docker-compose\n  - For Ubuntu (and Linux distros) - [Install docker-compose on Linux](https://docs.docker.com/compose/install/)\n  - For Windows and Mac\n\n## Steps\n- Clone the repo : https://github.com/kartik4949/AutoDeploy\n- Download a sample model and dependencies\n  - Run the command in a terminal from the AutoDeploy folder ``` wget https://github.com/kartik4949/AutoDeploy/files/7134516/model_dependencies.zip ```\n- Extract the zip folder to get a **model_dependencies** folder\n- Have your model ready \n  - Pickle file for scikitlearn\n  - ONNX model for Tensorflow,Pytorch, MXNet etc. [How to convert to ONNX model](https://github.com/onnx/tutorials)\n- Create the model [dependencies](https://github.com/kartik4949/AutoDeploy/wiki/Setup-Model-Dependencies)\n- Copy the dependencies over to a **model_dependencies** folder\n- Setup [configuration](https://google.com)\n- Steps for Docker deployment\n  - Build your docker image \n    - ```bash build.sh -r path/to/model/requirements.txt -c path/to/model/config.yaml```\n  - Start your containers\n    - ```bash start.sh -f path/to/config/file/in/autodeploy```\n- Steps for Kubernetes\n  - Build your docker image \n    - ```bash build.sh -r path/to/model/requirements.txt -c path/to/model/config.yaml```\n  - Apply kubeconfig files \n    - ``` kubectl -f k8s apply ```\n  - Print all pods\n    - ``` kubectl get pod ```\n  - Port forwarding of api and grafana service\n    - ``` kubectl port-forward autodeploy-pod-name 8000:8000 ```\n    - ``` kubectl port-forward grafana-pod-name 3000:3000 ```\n\n## Example (Docker deployment) - Iris Model Detection (Sci-Kit Learn).\n- Clone repo.\n- Dump your iris sklearn model via pickle, lets say `custom_model.pkl`.\n- Make a dir model_dependencies inside AutoDeploy.\n- Move `custom_model.pkl` to model_dependencies.\n- Create or import a reference `iris_reference.npy` file for data drift monitoring.\n  - Note: `iris_reference.npy` is numpy reference array used to find drift in incomming data.\n  - This reference data is usually in shape `(n, *shape_of_input)` e.g for iris data : np.zeros((100, 4))\n  - Shape (100, 4) means we are using 100 data points as reference for incomming input request.\n\n- Move `iris_reference.npy` to model_dependencies folder.\n- Refer below config file and make changes in configs/iris/config.yaml and save it.\n- Lastly make an empty reqs.txt file inside model_dependencies folder.\n```\nmodel:\n        model_type: 'sklearn'\n        model_path: 'custom_model.pkl' # Our model pickle file.\n        model_file_type: 'pickle'\n        version: '1.0.0'\n        model_name: 'sklearn iris detection model.'\n        endpoint: 'predict'\n        protected: 0\n        input_type: 'structured'\n        server:\n                name: 'autodeploy'\n                port: 8000\ndependency:\n        path: '/app/model_dependencies'\ninput_schema:\n        petal_length: 'float'\n        petal_width: 'float'\n        sepal_length: 'float'\n        sepal_width: 'float'\nout_schema:\n        out: 'int'\n        probablity: 'float'\n        status: 'int'\nmonitor:\n        server:\n                name: 'rabbitmq'\n                port: 5672\n        data_drift:\n                name: 'KSDrift'\n                reference_data: 'iris_reference.npy'\n                type: 'info'\n        metrics:\n                average_per_day:\n                        type: 'info'\n```\n- run ``` bash build.sh -r model_dependencies/reqs.txt -c  configs/iris/config.yaml```\n- run ``` bash start.sh -f configs/iris/config.yaml ```\n\nTada!! your model is deployed.\n\n## Example (Docker deployment) - Classification Detection\n\n- Clone repo.\n- Convert the model to Onnx file `model.onnx`.\n- Make a dir model_dependencies inside AutoDeploy.\n- Move `model.onnx` to model_dependencies.\n- Create or import a reference `classification_reference.npy` file for data drift monitoring.\n- Move `classification_reference.npy` to model_dependencies folder.\n- Refer below config file and make changes in configs/iris/config.yaml and save it.\n\n```\nmodel:\n        model_type: 'onnx'\n        model_path: 'horse_zebra.onnx'\n        model_file_type: 'onnx'\n        version: '1.0.0'\n        model_name: 'computer vision classification model.'\n        endpoint: 'predict'\n        protected: 0\n        input_type: 'serialized'\n        input_shape: [224, 224, 3]\n        server:\n                name: 'autodeploy'\n                port: 8000\npreprocess: 'custom_preprocess_classification'\ninput_schema:\n        input: 'string'\nout_schema:\n        out: 'int'\n        probablity: 'float'\n        status: 'int'\ndependency:\n        path: '/app/model_dependencies'\nmonitor:\n        server:\n                name: 'rabbitmq'\n                port: 5672\n        data_drift:\n                name: 'KSDrift'\n                reference_data: 'structured_ref.npy'\n                type: 'info'\n        custom_metrics: 'image_brightness'\n        metrics:\n                average_per_day:\n                        type: 'info'\n\n```\n- Make a reqs.txt file inside model_dependencies folder.\n- reqs.txt\n```\npillow\n```\n\n- Make preprocess.py\n```\nimport cv2\nimport numpy as np\n\nfrom register import PREPROCESS\n\n@PREPROCESS.register_module(name='custom_preprocess')\ndef iris_pre_processing(input):\n  return input\n\n@PREPROCESS.register_module(name='custom_preprocess_classification')\ndef custom_preprocess_fxn(input):\n  _channels = 3\n  _input_shape = (224, 224)\n  _channels_first = 1\n  input = cv2.resize(\n      input[0], dsize=_input_shape, interpolation=cv2.INTER_CUBIC)\n  if _channels_first:\n    input = np.reshape(input, (_channels, *_input_shape))\n  else:\n    input = np.reshape(input, (*_input_shape, _channels))\n  return np.asarray(input, np.float32)\n\n```\n- Make postproces.py\n\n```\nfrom register import POSTPROCESS\n\n@POSTPROCESS.register_module(name='custom_postprocess')\ndef custom_postprocess_fxn(output):\n  out_class, out_prob = output[0], output[1]\n  output = {'out': output[0],\n            'probablity': output[1],\n            'status': 200}\n  return output\n\n```\n- Make custom_metrics.py we will make a custom_metric to expose image_brightness\n```\nimport numpy as np\nfrom PIL import Image\nfrom register import METRICS\n\n\n@METRICS.register_module(name='image_brightness')\ndef calculate_brightness(image):\n  image = Image.fromarray(np.asarray(image[0][0], dtype='uint8'))\n  greyscale_image = image.convert('L')\n  histogram = greyscale_image.histogram()\n  pixels = sum(histogram)\n  brightness = scale = len(histogram)\n\n  for index in range(0, scale):\n    ratio = histogram[index] / pixels\n    brightness += ratio * (-scale + index)\n\n  return 1.0 if brightness == 255 else brightness / scale\n\n```\n- run ``` bash build.sh -r model_dependencies/reqs.txt -c configs/classification/config.yaml ```\n- run ``` bash start.sh -f configs/classification/config.yaml ```\n- To monitor the custom metric `image_brightness`: goto grafana and add panel to the dashboard with image_brightness as metric.\n\n\n## After deployment steps\n### Model Endpoint\n- http://address:port/endpoint is your model endpoint e.g http://localhost:8000/predict\n\n### Grafana \n- Open http://address:3000\n- Username and password both are `admin`.\n- Goto to add datasource.\n- Select first option prometheus.\n- Add http://prometheus:9090 in the source\n- Click save and test at bottom.\n- Goto dashboard and click import json file.\n- Upload dashboard/model.json avaiable in repository.\n- Now you have your dashboard ready!! feel free to add more panels with queries.\n\n## Preprocess\n- Add preprocess.py in model_dependencies folder\n- from register module import PROCESS register, to register your preprocess functions.\n```\nfrom register import PREPROCESS\n```\n- decorate your preprocess function with `@PREPROCESS.register_module(name='custom_preprocess')`\n```\n@PREPROCESS.register_module(name='custom_preprocess')\ndef function(input):\n  # process input\n  input = process(input)\n  return input\n```\n- Remeber we will use `custom_preprocess` name in our config file, add this in your config file.\n```\npreprocess: custom_preprocess\n```\n\n## Postprocess\n- Same as preprocess\n- Just remember schema of output from postprocess method should be same as definde in config file\n- i.e\n```\nout_schema:\n        out: 'int'\n        probablity: 'float'\n        status: 'int'\n```\n\n## Custom Metrics\n- from register import METRICS\n- register your function with METRIC decorator similar to preprocess\n- Example 1 : Simple single metric\n```\nimport numpy as np\nfrom PIL import Image\nfrom register import METRICS\n\n\n@METRICS.register_module(name='image_brightness')\ndef calculate_brightness(image):\n  image = Image.fromarray(np.asarray(image[0][0], dtype='uint8'))\n  greyscale_image = image.convert('L')\n  histogram = greyscale_image.histogram()\n  pixels = sum(histogram)\n  brightness = scale = len(histogram)\n\n  for index in range(0, scale):\n    ratio = histogram[index] / pixels\n    brightness += ratio * (-scale + index)\n\n  return 1.0 if brightness == 255 else brightness / scale\n\n```\n- We will use `image_brightness` in config file to expose this metric function.\n```\nmonitor:\n        server:\n                name: 'rabbitmq'\n                port: 5672\n        data_drift:\n                name: 'KSDrift'\n                reference_data: 'structured_ref.npy'\n                type: 'info'\n        custom_metrics: ['metric1', 'metric2']\n        metrics:\n                average_per_day:\n                        type: 'info'\n```\n- Example 2: Advance metric with multiple metrcis functions\n\n```\nimport numpy as np\nfrom PIL import Image\nfrom register import METRICS\n\n\n@METRICS.register_module(name='metric1')\ndef calculate_brightness(image):\n  return 1\n  \n@METRICS.register_module(name='metric2')\ndef metric2(image):\n  return 2\n\n```\n- config looks like\n```\nmonitor:\n        server:\n                name: 'rabbitmq'\n                port: 5672\n        data_drift:\n                name: 'KSDrift'\n                reference_data: 'structured_ref.npy'\n                type: 'info'\n        custom_metrics: ['metric1', 'metric2']\n        metrics:\n                average_per_day:\n                        type: 'info'\n```\n\n\u003c!-- LICENSE --\u003e\n## License\n\nDistributed under the MIT License. See `LICENSE` for more information.\n\n\u003c!-- CONTACT --\u003e\n## Contact\n\nKartik Sharma  - kartik4949@gmail.com\nNilav Ghosh - nilavghosh@gmail.com\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkartik4949%2Fautodeploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkartik4949%2Fautodeploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkartik4949%2Fautodeploy/lists"}