{"id":13578247,"url":"https://github.com/ebhy/budgetml","last_synced_at":"2025-05-15T13:07:31.116Z","repository":{"id":47583851,"uuid":"324778385","full_name":"ebhy/budgetml","owner":"ebhy","description":"Deploy a ML inference service on a budget in less than 10 lines of code.","archived":false,"fork":false,"pushed_at":"2024-02-12T17:29:24.000Z","size":1345,"stargazers_count":1342,"open_issues_count":4,"forks_count":63,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-04-14T20:58:54.666Z","etag":null,"topics":["api","data-science","deployment","fastapi","inference","machine-learning","mlops"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ebhy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-12-27T14:24:57.000Z","updated_at":"2025-04-09T05:27:32.000Z","dependencies_parsed_at":"2023-02-10T09:31:03.017Z","dependency_job_id":"c8b9f959-f042-45ab-8da4-b3b9156903a5","html_url":"https://github.com/ebhy/budgetml","commit_stats":{"total_commits":98,"total_committers":6,"mean_commits":"16.333333333333332","dds":"0.061224489795918324","last_synced_commit":"4667c77662125e63ef0733e81743bde6959f7339"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebhy%2Fbudgetml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebhy%2Fbudgetml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebhy%2Fbudgetml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ebhy%2Fbudgetml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ebhy","download_url":"https://codeload.github.com/ebhy/budgetml/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254346624,"owners_count":22055808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","data-science","deployment","fastapi","inference","machine-learning","mlops"],"created_at":"2024-08-01T15:01:28.757Z","updated_at":"2025-05-15T13:07:26.107Z","avatar_url":"https://github.com/ebhy.png","language":"Python","funding_links":[],"categories":["*Ops for AI","Python","LLMOps","其他_机器学习与深度学习","Model Serving"],"sub_categories":["Model Serving \u0026 Inference","Observability","Popular-LLM"],"readme":"\n\n\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"docs/static/header_new.png\"\u003e\n\n\u003ch1\u003eBudgetML: Deploy ML models on a budget\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e •\n  \u003ca href=\"#quickstart\"\u003eQuickstart\u003c/a\u003e •\n  \u003ca href=\"https://github.com/ebhy/budgetml/discussions\"\u003eCommunity\u003c/a\u003e  •\n  \u003ca href=\"https://github.com/ebhy/budgetml/tree/main/docs\"\u003eDocs\u003c/a\u003e \n\u003c/p\u003e\n\n[![PyPI - ZenML Version](https://img.shields.io/pypi/v/budgetml.svg?label=pip\u0026logo=PyPI\u0026logoColor=white)](https://pypi.org/project/budgetml/)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/budgetml)](https://pypi.org/project/budgetml/)\n[![PyPI Status](https://pepy.tech/badge/budgetml)](https://pepy.tech/project/budgetml)\n![GitHub](https://img.shields.io/github/license/ebhy/budgetml)\n\u003c/div\u003e\n\n\n\u003cdiv align=\"center\" style=\"font-size:30px\"\u003e \n    \u003cb\u003eNotice: This library is not being actively maintained, and we're looking for someone to update it and keep it going! Reach out to me \n    directly if you would like to help!\u003c/b\u003e\n\u003c/div\u003e\n\n# Why\nBudgetML is perfect for practitioners who would like to quickly deploy their models to an endpoint, but not waste a lot of \ntime, money, and effort trying to figure out how to do this end-to-end.\n\nWe built BudgetML because it's hard to find a simple way to get a model in production **fast** and **cheaply**. \n\n* Cloud functions are limited in memory and cost a lot at scale. \n\n* Kubernetes clusters are overkill for one single model. \n\n* Deploying from scratch involves learning too many different concepts like SSL certificate generation, Docker, REST, Uvicorn/Gunicorn, backend servers etc., that are simply not within the scope of a typical data scientist.\n\nBudgetML is our answer to this challenge. It is supposed to be fast, easy, and developer-friendly. It is by no means meant to be used in a full-fledged \nproduction-ready setup. It is simply a means to get a server up and running **as fast as possible** with the **lowest costs possible**.\n\n\nBudgetML lets you deploy your model on a [Google Cloud Platform preemptible instance](https://cloud.google.com/compute/docs/instances/preemptible) \n(which is **~80% cheaper** than a regular instance) with a **secured HTTPS API** endpoint.\nThe tool sets it up in a way that the instance autostarts when it shuts down (at least once every 24 hours) with **only a few minutes of downtime**.\nBudgetML ensures the cheapest possible API endpoint with the lowest possible downtime. \n\n\n## Key Features\n* Automatic [FastAPI](https://fastapi.tiangolo.com/) server endpoint generation (it's faster than Flask).\n* Fully interactive docs via [Swagger](https://swagger.io/docs/).\n* Built-in SSL certificate generation via [LetsEncrypt](https://letsencrypt.org/) and [docker-swag](https://github.com/linuxserver/docker-swag).\n* Uses cheap preemtible instances but has 99% uptime!\n* Complete OAuth2 secured endpoints with [Password and Bearer pattern](https://fastapi.tiangolo.com/tutorial/security/simple-oauth2/).\n\n## Cost comparison\nBudgetML uses Google Cloud Preemptible instances under-the-hood to reduce costs by 80%. This can potentially mean hundreds of dollars \nworth of savings. Here is a screenshot of the `e2-highmem` GCP series, which is regular family of instances to be using for \nmemory intense tasks like ML model inference functions. See the following price comparison (as of Jan 31, 2021 [[source](https://cloud.google.com/compute/vm-instance-pricing)])\n\n![GCP costs](docs/static/gcp_costs.png)\n\nEven with the lowest machine_type, there is a **$46/month** savings, and with the highest configuration this is **$370/month** savings!\n\n## Installation\nBudgetML is available for easy installation into your environment via PyPI:\n```bash\npip install budgetml\n```\nAlternatively, if you’re feeling brave, feel free to install the bleeding edge:\n\n**NOTE:** Do so at your own risk; no guarantees given!\n```bash\npip install git+https://github.com/ebhy/budgetml.git@main --upgrade\n```\n\n## Quickstart\nBudgetML aims for as simple a process as possible. First set up a predictor:\n\n```python\n# predictor.py\nclass Predictor:\n    def load(self):\n        from transformers import pipeline\n        self.model = pipeline(task=\"sentiment-analysis\")\n\n    async def predict(self, request):\n        # We know we are going to use the `predict_dict` method, so we use\n        # the request.payload pattern\n        req = request.payload\n        return self.model(req[\"text\"])[0]\n```\n\nThen launch it with a simple script:\n```python\n# deploy.py\nimport budgetml\nfrom predictor import Predictor\n\n# add your GCP project name here.\nbudgetml = budgetml.BudgetML(project='GCP_PROJECT')\n\n# launch endpoint\nbudgetml.launch(\n    Predictor,\n    domain=\"example.com\",\n    subdomain=\"api\",\n    static_ip=\"32.32.32.322\",\n    machine_type=\"e2-medium\",\n    requirements=['tensorflow==2.3.0', 'transformers'],\n)\n```\nFor a deeper dive, [check out the detailed guide](examples/deploy_simple_model) in the [examples](examples) directory. For \nmore information about the BudgetML API, refer to the [docs](docs).\n\n## Screenshots\nInteractive docs to test endpoints. Support for Images.\n![Interactive docs](docs/static/swagger_ui.png)\n\nPassword-protected endpoints:\n![Password protected endpoints](docs/static/swagger_password_auth.png)\n\nSimple prediction interface:\n![Simple Prediction Interface of BudgetML](docs/static/swagger_predict_dict.png)\n\n## Projects using BudgetML\nWe are proud that BudgetML is actively being used in the following live products:\n\n* [PicHance](https://pichance.com)\n* [you-tldr](https://you-tldr.com)\n\n## ZenML: For production scenarios\nBudgetML is for users on a budget. If you're working in a more serious production environment, then consider using \n[ZenML](https://github.com/zenml-io/zenml) as the perfect open-source MLOps framework for ML production needs. It does \nmore than just deployments, and is more suited for professional workplaces.\n\n## Proudly built by two brothers\nWe are two brothers who love building products, especially ML-related products that make life easier for people.\nIf you use this tool for any of your products, we would love to hear about it and potentially add it to this space.\nPlease get in touch [via email](mailto:htahir111@gmail.com). \n\nOh and please do consider giving us a GitHub star if you like the repository - open-source is hard, and the support \nkeeps us going.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febhy%2Fbudgetml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Febhy%2Fbudgetml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Febhy%2Fbudgetml/lists"}