{"id":26123237,"url":"https://github.com/aws-samples/easy-model-deployer","last_synced_at":"2025-04-13T14:21:02.952Z","repository":{"id":278670954,"uuid":"922125508","full_name":"aws-samples/easy-model-deployer","owner":"aws-samples","description":"A user-friendly Command-line/SDK tool that makes it quickly and easier to deploy open-source LLMs on AWS","archived":false,"fork":false,"pushed_at":"2025-04-03T16:01:33.000Z","size":41465,"stargazers_count":33,"open_issues_count":3,"forks_count":5,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-03T16:16:35.984Z","etag":null,"topics":["comfyui-workflow","deepseek","deepseek-r1","ec2","ecs","gemma3","huggingface","inferentia-2","internlm2","langchain","large-language-model","ollama","openai-compatible-api","qwen2-5","qwq","qwq-32b","sagemaker","vllm"],"latest_commit_sha":null,"homepage":"https://aws-samples.github.io/easy-model-deployer/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/aws-samples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-01-25T11:40:31.000Z","updated_at":"2025-04-03T02:11:13.000Z","dependencies_parsed_at":"2025-03-24T14:38:12.942Z","dependency_job_id":null,"html_url":"https://github.com/aws-samples/easy-model-deployer","commit_stats":null,"previous_names":["aws-samples/easy-model-deployer"],"tags_count":11,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Feasy-model-deployer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Feasy-model-deployer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Feasy-model-deployer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/aws-samples%2Feasy-model-deployer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/aws-samples","download_url":"https://codeload.github.com/aws-samples/easy-model-deployer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248725092,"owners_count":21151626,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["comfyui-workflow","deepseek","deepseek-r1","ec2","ecs","gemma3","huggingface","inferentia-2","internlm2","langchain","large-language-model","ollama","openai-compatible-api","qwen2-5","qwq","qwq-32b","sagemaker","vllm"],"created_at":"2025-03-10T15:46:10.033Z","updated_at":"2025-04-13T14:21:02.939Z","avatar_url":"https://github.com/aws-samples.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n    \u003ch3 align=\"center\"\u003eEasy Model Deployer: Simple, Efficient, and Easy-to-Integrate\u003c/h3\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://aws-samples.github.io/easy-model-deployer/en/installation\"\u003e\u003cstrong\u003eDocumentation\u003c/strong\u003e\u003c/a\u003e ·\n  \u003ca href=\"https://github.com/aws-samples/easy-model-deployer/releases\"\u003e\u003cstrong\u003eChangelog\u003c/strong\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://opensource.org/licenses/MIT\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-MIT-yellowgreen.svg\" alt=\"MIT License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/easy_model_deployer\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/easy_model_deployer.svg?logo=pypi\u0026label=PyPI\u0026logoColor=gold\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/easy_model_deployer\"\u003e\u003cimg alt=\"PyPI - Downloads\" src=\"https://img.shields.io/pypi/dw/easy-model-deployer\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/aws-samples/easy-model-deployer/actions/workflows/release-package.yml\"\u003e\u003cimg src=\"https://github.com/aws-samples/easy-model-deployer/actions/workflows/release-package.yml/badge.svg\" alt=\"Build Status\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n## 🔥 Latest News\n\n- 2025-03-17: Deploy Gemma 3 series models with [one command line](https://github.com/aws-samples/easy-model-deployer/blob/main/docs/en/best_deployment_practices.md##famous-models###gemma-3-series).\n- 2025-03-06: Deploy QwQ-32B with [one command line](docs/en/best_deployment_practices.md##famous-models###qwen-series###qwq-32b).\n\n## Introduction\n\nEasy Model Deployer is a lightweight tool designed to simplify the deployment of **Open-Source LLMs** ([Supported Models](docs/en/supported_models.md)) and Custom Models. It provides **OpenAI's Completions API** and [**LangChain Interface**](https://github.com/langchain-ai/langchain). Built for developers who need reliable and scalable model serving without complex setup, it seamlessly integrates with AWS services for efficient model deployment.\n\n![cli](docs/images/cli-all.gif)\n\n**Key Features**\n\n- One-click deployment of models to the cloud (Amazon SageMaker, Amazon ECS, Amazon EC2)\n- Diverse model types (LLMs, VLMs, Embeddings, Vision, etc.)\n- Rich inference engine (vLLM, TGI, Lmdeploy, etc.)\n- Different instance types (CPU/GPU/AWS Inferentia)\n- Convenient integration (OpenAI Compatible API, LangChain client, etc.)\n\n## 🚀 Quick Install\n\nInstall Easy Model Deployer with `pip`, currently support for Python 3.9 and above:\n\n```bash\npip install easy_model_deployer  # or pip3 install easy_model_deployer\nemd\n```\n\nVisit our [documentation](https://aws-samples.github.io/easy-model-deployer/en/installation/) to learn more.\n\n## 🔧 Usage\n\n### (Optional) Configure AWS Profile\n\n\u003e If you have already configured your AWS credentials using the AWS CLI, you can skip this step.\n\nYou can configure the AWS profile by EMD. If you don't set any AWS profile, the EMD will use the default credentials in your terminal, you can also configure your credentials by [`aws configure`](https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html#cli-configure-files-methods).\n\n```bash\nemd config set-default-profile-name\n```\n\n### Bootstrap\n\nPrepare the essential resources required for model deployment.\n\nFor more information, please refer to [Architecture](https://aws-samples.github.io/easy-model-deployer/en/architecture/).\n\n```bash\nemd bootstrap\n```\n\n\u003e **💡 Tip** Once you upgrade the EMD by `pip`, you need to run this command again.\n\n### Deploy Model\n\nDeploy models with an interactive CLI or one command.\n\n```bash\nemd deploy\n```\n\n![deploy](docs/images/cli-deploy.gif)\n\n\u003e **💡 Tip** To view all available parameters, run `emd deploy --help`.\n\u003e When you see the message \"Waiting for model: ...\", it means the deployment task has started and you can stop the terminal output by pressing `Ctrl+C`.\n\u003e\n\u003e - For more information on deployment parameters, please refer to the [Deployment parameters](docs/en/installation.md).\n\u003e - For best practice examples of using command line parameters, please refer to the [Best Deployment Practices](docs/en/best_deployment_practices.md).\n\n### Show Status\n\nCheck the status of the model deployment task.\n\n```bash\nemd status\n```\n\n\u003e **💡 Tip** The EMD allows launch multiple deployment tasks simultaneously.\n\n### Quick Invocation\n\nInvoke the deployed model for testing by CLI.\n\n```bash\nemd invoke DeepSeek-R1-Distill-Qwen-1.5B\n```\n\n\u003e **💡 Tip** You can find the *ModelId* in the output by `emd status`.\n\n- [Integration examples](https://aws-samples.github.io/easy-model-deployer/)\n- [EMD client](docs/en/emd_client.md)\n- [Langchain interface](docs/en/langchain_interface.md)\n- [OpenAI compatible interface](docs/en/openai_compatiable.md).\n\n\u003e **💡 Tip** OpenAI Compatible API is supported only for Amazon ECS and Amazon EC2 deployment types.\n\n### List Supported Models\n\nQuickly see what models are supported, this command will output all information related to deployment. (Plese browse [Supported Models](docs/en/supported_models.md) for more information.)\n\n```bash\nemd list-supported-models\n```\n\nThe following command is recommended to just list the model types.\n\n```bash\nemd list-supported-models | jq -r '.[] | \"\\(.model_id)\\t\\(.model_type)\"' | column -t -s $'\\t' | sort\n```\n\n### Delete Model\n\nDelete the deployed model.\n\n```bash\nemd destroy DeepSeek-R1-Distill-Qwen-1.5B\n```\n\n\u003e **💡 Tip** You can find the *ModelId* in the output by `emd status`.\n\n## 📖 Documentation\n\nFor advanced configurations and detailed guides, visit our [documentation site](https://aws-samples.github.io/easy-model-deployer/).\n\n## 🤝 Contributing\n\nWe welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws-samples%2Feasy-model-deployer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Faws-samples%2Feasy-model-deployer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Faws-samples%2Feasy-model-deployer/lists"}