{"id":19748000,"url":"https://github.com/tensorchord/openmodelz-charts","last_synced_at":"2026-03-19T11:27:16.478Z","repository":{"id":183368183,"uuid":"670030924","full_name":"tensorchord/openmodelz-charts","owner":"tensorchord","description":"Helm chart for OpenModelZ","archived":false,"fork":false,"pushed_at":"2023-10-12T02:07:24.000Z","size":36,"stargazers_count":2,"open_issues_count":1,"forks_count":1,"subscribers_count":5,"default_branch":"gh-pages","last_synced_at":"2025-01-10T21:11:42.670Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://tensorchord.github.io/openmodelz-charts/index.yaml","language":"Smarty","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tensorchord.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-07-24T06:29:09.000Z","updated_at":"2023-08-04T01:56:40.000Z","dependencies_parsed_at":"2024-01-30T00:10:04.892Z","dependency_job_id":null,"html_url":"https://github.com/tensorchord/openmodelz-charts","commit_stats":null,"previous_names":["tensorchord/openmodelz-charts"],"tags_count":15,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorchord%2Fopenmodelz-charts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorchord%2Fopenmodelz-charts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorchord%2Fopenmodelz-charts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tensorchord%2Fopenmodelz-charts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tensorchord","download_url":"https://codeload.github.com/tensorchord/openmodelz-charts/tar.gz/refs/heads/gh-pages","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241078539,"owners_count":19905877,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T02:19:43.800Z","updated_at":"2026-02-26T02:08:24.710Z","avatar_url":"https://github.com/tensorchord.png","language":"Smarty","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n# OpenModelZ\n\nHelm chart for OpenModelZ\n\n\u003c/div\u003e\n\n\u003cp align=center\u003e\n\u003ca href=\"https://discord.gg/KqswhpVgdU\"\u003e\u003cimg alt=\"discord invitation link\" src=\"https://dcbadge.vercel.app/api/server/KqswhpVgdU?style=flat\"\u003e\u003c/a\u003e\n\u003ca href=\"https://twitter.com/TensorChord\"\u003e\u003cimg src=\"https://img.shields.io/twitter/follow/tensorchord?style=social\" alt=\"trackgit-views\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://docs.open.modelz.ai\"\u003e\u003cimg src=\"https://img.shields.io/badge/docs.open.modelz.ai-455946.svg?style=socail\u0026logo=googlechrome\u0026logoColor=white\" alt=\"docs\" /\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## Why use OpenModelZ\n\nOpenModelZ is the ideal solution for practitioners who want to quickly deploy their machine learning models to a (public or private) endpoint without the hassle of spending excessive time, money, and effort to figure out the entire end-to-end process.\n\nWe created OpenModelZ in response to the difficulties of finding a simple, cost-effective way to get models into production fast. Traditional deployment methods can be complex and time-consuming, requiring significant effort and resources to get models up and running.\n\n- Kubernetes: Setting up and maintaining Kubernetes and Kubeflow can be challenging due to their technical complexity. Data scientists spend significant time configuring and debugging infrastructure instead of focusing on model development.\n- Managed services: Alternatively, using a managed service like AWS SageMaker can be expensive and inflexible, limiting the ability to customize deployment options.\n- Virtual machines: As an alternative, setting up a cloud VM-based solution requires learning complex infrastructure concepts like load balancers, ingress controllers, and other components. This takes a lot of specialized knowledge and resources.\n\nWith OpenModelZ, we take care of the underlying technical details for you, and provide a simple and easy-to-use CLI to deploy your models to **any cloud (GCP, AWS, or others), your home lab, or even a single machine**.\n\nYou could **start from a single machine and scale it up to a cluster of machines** without any hassle. OpenModelZ lies at the heart of our [ModelZ](https://modelz.ai), which is a serverless inference platform. It's used in production to deploy models for our customers.\n\n## Documentation 📝\n\nYou can find the documentation at [docs.open.modelz.ai](https://docs.open.modelz.ai).\n\n## Quick Start 🚀\n\nOnce you've installed the `mdz` you can start deploying models and experimenting with them.\n\nThere are only two concepts in `mdz`:\n\n- **Deployment**: A deployment is a running inference service. You could configure the number of replicas, the port, and the image, and some other parameters.\n- **Server**: A server is a machine that could run the deployments. It could be a cloud VM, a PC, or even a Raspberry Pi. You could start from a single server and scale it up to a cluster of machines without any hassle.\n\n### Bootstrap `mdz`\n\nIt's super easy to bootstrap the `mdz` server. You just need to find a server (could be a cloud VM, a home lab, or even a single machine) and run the `mdz server start` command. The `mdz` server will be bootstrapped on the server and you could start deploying your models.\n\n```\n$ mdz server start\n🚧 Creating the server...\n🚧 Initializing the load balancer...\n🚧 Initializing the GPU resource...\n🚧 Initializing the server...\n🚧 Waiting for the server to be ready...\n🐋 Checking if the server is running...\nAgent:\n Version:       v0.0.13\n Build Date:    2023-07-19T09:12:55Z\n Git Commit:    84d0171640453e9272f78a63e621392e93ef6bbb\n Git State:     clean\n Go Version:    go1.19.10\n Compiler:      gc\n Platform:      linux/amd64\n🐳 The server is running at http://192.168.71.93.modelz.live\n🎉 You could set the environment variable to get started!\n\nexport MDZ_URL=http://192.168.71.93.modelz.live\n```\n\nThe internal IP address will be used as the default endpoint of your deployments. You could provide the public IP address of your server to the `mdz server start` command to make it accessible from the outside world.\n\n```bash\n# Provide the public IP as an argument\n$ mdz server start 1.2.3.4\n```\n\n### Create your first deployment\n\nOnce you've bootstrapped the `mdz` server, you can start deploying your first applications.\n\n```\n$ mdz deploy --image aikain/simplehttpserver:0.1 --name simple-server --port 80\nInference simple-server is created\n$ mdz list\n NAME           ENDPOINT                                                          STATUS  INVOCATIONS  REPLICAS \n simple-server  http://simple-server-4k2epq5lynxbaayn.192.168.71.93.modelz.live   Ready             2  1/1      \n                http://192.168.71.93.modelz.live/inference/simple-server.default                                 \n```\n\nYou could access the deployment by visiting the endpoint URL. It will be `http://simple-server-4k2epq5lynxbaayn.192.168.71.93.modelz.live` in this case. The endpoint could be accessed from the outside world as well if you've provided the public IP address of your server to the `mdz server start` command.\n\n### Scale your deployment\n\nYou could scale your deployment by using the `mdz scale` command.\n\n```bash\n$ mdz scale simple-server --replicas 3\n```\n\nThe requests will be load balanced between the replicas of your deployment.\n\n### Debug your deployment\n\nSometimes you may want to debug your deployment. You could use the `mdz logs` command to get the logs of your deployment.\n\n```bash\n$ mdz logs simple-server\nsimple-server-6756dd67ff-4bf4g: 10.42.0.1 - - [27/Jul/2023 02:32:16] \"GET / HTTP/1.1\" 200 -\nsimple-server-6756dd67ff-4bf4g: 10.42.0.1 - - [27/Jul/2023 02:32:16] \"GET / HTTP/1.1\" 200 -\nsimple-server-6756dd67ff-4bf4g: 10.42.0.1 - - [27/Jul/2023 02:32:17] \"GET / HTTP/1.1\" 200 -\n```\n\nYou could also use the `mdz exec` command to execute a command in the container of your deployment. You do not need to ssh into the server to do that.\n\n```\n$ mdz exec simple-server ps\nPID   USER     TIME   COMMAND\n    1 root       0:00 /usr/bin/dumb-init /bin/sh -c python3 -m http.server 80\n    7 root       0:00 /bin/sh -c python3 -m http.server 80\n    8 root       0:00 python3 -m http.server 80\n    9 root       0:00 ps\n$ mdz exec simple-server -ti bash\nbash-4.4# uname -r\n5.19.0-46-generic\nbash-4.4# \n```\n\nOr you could port-forward the deployment to your local machine and debug it locally.\n\n```\n$ mdz port-forward simple-server 7860\nForwarding inference simple-server to local port 7860\n```\n\n### Add more servers\n\nYou could add more servers to your cluster by using the `mdz server join` command. The `mdz` server will be bootstrapped on the server and join the cluster automatically.\n\n```\n$ mdz server list\n NAME   PHASE  ALLOCATABLE      CAPACITY        \n node1  Ready  cpu: 16          cpu: 16         \n               mem: 32784748Ki  mem: 32784748Ki \n node2  Ready  cpu: 16          cpu: 16         \n               mem: 32784748Ki  mem: 32784748Ki \n```\n\n### Label your servers\n\nYou could label your servers to deploy your models to specific servers. For example, you could label your servers with `gpu=true` and deploy your models to servers with GPUs.\n\n```\n$ mdz server label node3 gpu=true type=nvidia-a100\n$ mdz deploy --image aikain/simplehttpserver:0.1 --name simple-server --port 80 --node-labels gpu=true,type=nvidia-a100\n```\n\n## Roadmap 🗂️\n\nPlease checkout [ROADMAP](https://docs.open.modelz.ai/community).\n\n## Contribute 😊\n\nWe welcome all kinds of contributions from the open-source community, individuals, and partners.\n\n- Join our [discord community](https://discord.gg/KqswhpVgdU)!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorchord%2Fopenmodelz-charts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftensorchord%2Fopenmodelz-charts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftensorchord%2Fopenmodelz-charts/lists"}