{"id":19808802,"url":"https://github.com/premai-io/serverless-examples","last_synced_at":"2025-08-19T15:08:47.340Z","repository":{"id":229035961,"uuid":"771611704","full_name":"premAI-io/serverless-examples","owner":"premAI-io","description":"🚀 End-to-end examples and analysis of deploying LLMs serverless using Modal, Runpod, and Beam","archived":false,"fork":false,"pushed_at":"2024-03-25T16:09:15.000Z","size":71,"stargazers_count":27,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-05-01T07:37:49.613Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/premAI-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-03-13T16:08:06.000Z","updated_at":"2024-11-19T06:57:17.000Z","dependencies_parsed_at":"2025-05-01T07:33:00.218Z","dependency_job_id":"4df7a682-0b32-4e48-9c6b-bbd9afee2cdc","html_url":"https://github.com/premAI-io/serverless-examples","commit_stats":null,"previous_names":["premai-io/serverless-examples"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/premAI-io/serverless-examples","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/premAI-io%2Fserverless-examples","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/premAI-io%2Fserverless-examples/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/premAI-io%2Fserverless-examples/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/premAI-io%2Fserverless-examples/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/premAI-io","download_url":"https://codeload.github.com/premAI-io/serverless-examples/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/premAI-io%2Fserverless-examples/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271173371,"owners_count":24711667,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-19T02:00:09.176Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-12T09:14:49.404Z","updated_at":"2025-08-19T15:08:47.311Z","avatar_url":"https://github.com/premAI-io.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless LLM Deployment Examples\n\nWelcome to the repository containing a set of hackable examples for serverless deployement of Large Language Models (LLMs). Here, we explore and analyze three services: [Modal Labs](https://modal.com/), [Beam Cloud](https://www.beam.cloud/), and [Runpod](https://www.runpod.io/), each abstracting out the deployment process at different levels.\n\n| Service       | Blogpost                                                    | Implementation   |\n|---------------|-------------------------------------------------------------|------------------|\n| Modal Labs    | [Tutorial Blogpost](https://blog.premai.io/serverless-deployment-using-huggingface-and-modal/) | [Modal Labs Deployment](/deploy_modal/) |\n| Beam Cloud    | [Tutorial Blogpost](https://blog.premai.io/deploy-google-gemma-serverless-using-beam-cloud/) | [Beam Cloud Deployment](/deploy_beam/) |\n| RunPod        | [Tutorial Blogpost](https://blog.premai.io/serverless-deploy-mistral-2-7b-runpod/) | [RunPod Deployment](/deploy_runpod/) |\n\nWe provide blog posts for each service, as well as dedicated repositories containing full code examples and instructions on how to run and test them.\n\n## Test Deployed Model\n\nIf you've followed our tutorials and deployed your models using any of the mentioned services, you can test the deployments from here. Please note that testing is currently available only for streaming. However, if you want to make changes, feel free to do so. Before getting started, please install the requirements from here.\n\n```bash\npip install -r requirements.txt\n```\n\nNow, assuming you deployed your model using either of the services, you can run `test.py` as shown below:\n\nFor Modal and Beam Cloud:\n```bash\npython3 test.py modal --url \u003cYOUR-DEPLOYED-MODEL/BEAM-URL\u003e --prompt \"hello\"\n```\n\nFor RunPod, you also need to provide the service ID:\n```bash\npython3 test.py modal \\\n    --url \u003cYOUR-DEPLOYED-RUNPOD-URL\u003e \\\n    --prompt \"hello\" \\\n    --runpod_id \u003cRUNPOD-ID\u003e\n```\n\nReplace `\u003cRUNPOD-ID\u003e` with a value that looks like this: `80r0eh3jel99f8` (this is an example ID).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpremai-io%2Fserverless-examples","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpremai-io%2Fserverless-examples","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpremai-io%2Fserverless-examples/lists"}