{"id":22656517,"url":"https://github.com/dhanushpittala11/huggingfacemodeldeployment_aws_sagemaker","last_synced_at":"2026-05-05T14:02:38.006Z","repository":{"id":266994876,"uuid":"899976967","full_name":"dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker","owner":"dhanushpittala11","description":"This project is about deploying HuggingFace model(distilbert-base-uncased-distilled-squad) on AWS sagemaker for the task of question Answering. The model is configured ,IAM role is created for permissions to create endpoint and the model is deployed to Sagemaker Inference.","archived":false,"fork":false,"pushed_at":"2024-12-08T05:54:36.000Z","size":994,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-04T12:36:24.010Z","etag":null,"topics":["aws","huggingface-transformers","jupyterlab","python","pytorch","question-answering","s3-bucket","s3-storage","sagemaker","sagemaker-deployment","sagemaker-studio","sagemakerendpoint"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dhanushpittala11.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-07T14:33:08.000Z","updated_at":"2024-12-08T05:54:39.000Z","dependencies_parsed_at":"2025-06-03T23:42:39.923Z","dependency_job_id":"136fcf29-fe67-41cc-8acf-c3f1320aadee","html_url":"https://github.com/dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker","commit_stats":null,"previous_names":["dhanushpittala11/huggingfacemodeldeployment_aws_sagemaker"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhanushpittala11%2FHuggingFaceModelDeployment_AWS_Sagemaker","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhanushpittala11%2FHuggingFaceModelDeployment_AWS_Sagemaker/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhanushpittala11%2FHuggingFaceModelDeployment_AWS_Sagemaker/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhanushpittala11%2FHuggingFaceModelDeployment_AWS_Sagemaker/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dhanushpittala11","download_url":"https://codeload.github.com/dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dhanushpittala11%2FHuggingFaceModelDeployment_AWS_Sagemaker/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269931487,"owners_count":24498724,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-11T02:00:10.019Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","huggingface-transformers","jupyterlab","python","pytorch","question-answering","s3-bucket","s3-storage","sagemaker","sagemaker-deployment","sagemaker-studio","sagemakerendpoint"],"created_at":"2024-12-09T10:14:44.454Z","updated_at":"2026-05-05T14:02:32.961Z","avatar_url":"https://github.com/dhanushpittala11.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HuggingFace Model Deployment on AWS Sagemaker\n\n## Table of Contents\n * [Introduction](#introduction)\n * [Setup of AWS Sagemaker Studio](#setup-of-aws-sagemaker-studio)\n * [Run and Deploy the model](#run-and-deploy-the-model)\n * [Test Inference](#test--inference)\n * [Bug / Feature Request](#bug---feature-request)\n * [Techstack Used](#techstack-used)\n * [License](#license)\n\n## Introduction\n  This project is about using a huggingface model on the task of Question Answering and Deploying it on AWS Sagemaker Studio. First we create a bucket in AWS S3 where the model is stored. Then we configure for loading the model from the hub. Create IAM role with permissions to create an endpoint. Then the model is deployed to Sagemaker Inference.\n\n## Setup of AWS Sagemaker Studio\n1. Login to your AWS Account. Create a new one if you don't have account.\n2. Open **AWS Sagemaker AI** service.\n3. Set up Domain\n4. A user profile will be created automatically\n    ![](launch_jupyterlab.png)\n   Now, launch the Sagemaker Studio as shown here.\n5. In the Sagemaker Studio, select Instance type and then click on Run Lab. You will be directed to the JupyterLab notebook.\n    ![](Run_jupyterLab.png)\n\n## Run and Deploy the model\n  Upload the [notebook](hf.ipynb) on the environment and run it. Choose the instance type as per the availability in your zone.\n\n## Test Inference\n  1. On the Sagemaker Studio, go to Deployments section and click on Endpoints. The page will show Endpoint Summary. Below that, click on \n     Test inference to perform the inference.\n  2. Under the JSON column, give the input in the form of question and context. For example-\n     ``` bash\n     {\n      \"body\": {\n        \"inputs\": {\n          \"question\": \"How many indeginious territories exist in amazon rainforest?\",\n          \"context\": \"The Amazon rainforest,also called Amazon jungle or Amazonia, is a moist broadleaf tropical rainforest in the Amazon \n           biome \n           that covers most of the Amazon basin of South America. This basin encompasses 7,000,000 km2 (2,700,000 sq mi), of which 6,000,000 \n            km2 \n           (2,300,000 sq mi) are covered by the rainforest. This region includes territory belonging to nine nations and 3,344 indigenous \n            territories.\"\n                }\n              },\n           \"contentType\": \"application/json\",\n        \"endpointName\": \"huggingface-pytorch-inference-2024-12-07-13-31-37-510\"\n      }\n      ```\n     Refer to this image for reference.\n      ![](deploymentInference1.png) \n      \n     Refer to this image for the Inference result. The answer to the question is indicated by **\"answer\": \"3,344\"** meaning that the number \n     of indeginious territories is 3344.\n\n\n     ![](DeploymentInference2.png)\n\n## Bug / Feature Request\n  If you find a bug (the website couldn't handle the query and / or gave undesired results), kindly open an issue [here](https://github.com/dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker/issues/new) by including your search query and the expected \n   result.\n\n  If you'd like to request a new function, feel free to do so by opening an issue [here](https://github.com/dhanushpittala11/HuggingFaceModelDeployment_AWS_Sagemaker/issues/new). Please include sample queries and their corresponding results.\n\n## Techstack Used\n  ### Cloud and Storage\n   #### AWS Services:\n   * **Amazon SageMaker:** Used for deploying and managing machine learning models.\n   * **Amazon S3:** Default bucket for storing data, models, and logs.\n   * **IAM (Identity and Access Management):** For managing roles and permissions.\n  ### Machine Learning Frameworks\n   #### Hugging Face Transformers:\n   * **Model:** distilbert-base-uncased-distilled-squad (question-answering task). Deployed using the Hugging Face Model class in SageMaker.\n   * **PyTorch:** Backend framework for running the Transformer model.\n  ### Programming\n   * **Python:** Language for implementing the solution.\n   * **JupyterLab/Jupyter Notebooks:** Likely used for development and testing in SageMaker Studio.\n  ### Model Deployment\n   * **SageMaker Inference:** Deployment of the Hugging Face model to an endpoint for inference.\n   * **Instance Type:** ml.m5.xlarge\n  ### API Integration\n   * **Predict API:** Example request defined with inputs (question and context) for performing inference.\n  ### Development Tools\n   * **SageMaker Studio:** Integrated development environment for working with SageMaker.\n   * **Boto3:** AWS SDK for Python to interact with IAM for role setup.\n## License\n                       GNU GENERAL PUBLIC LICENSE\n                       Version 3, 29 June 2007\n\n Copyright (C) 2007 Free Software Foundation, Inc. \u003chttps://fsf.org/\u003e\n Everyone is permitted to copy and distribute verbatim copies\n of this license document, but changing it is not allowed.\n\n                            Preamble\n\n  The GNU General Public License is a free, copyleft license for\nsoftware and other kinds of works.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhanushpittala11%2Fhuggingfacemodeldeployment_aws_sagemaker","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdhanushpittala11%2Fhuggingfacemodeldeployment_aws_sagemaker","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdhanushpittala11%2Fhuggingfacemodeldeployment_aws_sagemaker/lists"}