{"id":16754380,"url":"https://github.com/ritchie46/serverless-model-aws","last_synced_at":"2025-04-10T16:13:26.919Z","repository":{"id":71707591,"uuid":"148633821","full_name":"ritchie46/serverless-model-aws","owner":"ritchie46","description":"Deploy any Machine Learning model serverless in AWS.","archived":false,"fork":false,"pushed_at":"2018-10-17T09:39:17.000Z","size":27,"stargazers_count":24,"open_issues_count":0,"forks_count":3,"subscribers_count":2,"default_branch":"master-vik","last_synced_at":"2025-03-24T14:05:14.574Z","etag":null,"topics":["aws","aws-lambda","deployment","serverless"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ritchie46.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-13T12:24:23.000Z","updated_at":"2025-02-28T15:24:26.000Z","dependencies_parsed_at":"2023-05-15T23:00:16.273Z","dependency_job_id":null,"html_url":"https://github.com/ritchie46/serverless-model-aws","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ritchie46%2Fserverless-model-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ritchie46%2Fserverless-model-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ritchie46%2Fserverless-model-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ritchie46%2Fserverless-model-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ritchie46","download_url":"https://codeload.github.com/ritchie46/serverless-model-aws/tar.gz/refs/heads/master-vik","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248251480,"owners_count":21072689,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","deployment","serverless"],"created_at":"2024-10-13T03:04:46.546Z","updated_at":"2025-04-10T16:13:26.904Z","avatar_url":"https://github.com/ritchie46.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Model Endpoint and Serverless Batch Transform Job on AWS\n\nThis repo is a basis template for deploying a model as a batch transform job or as a server endpoint on AWS Elastic Container Service.\n\nFor a more elaborate tutorial, [read my blog post](https://www.ritchievink.com/blog/2018/09/16/deploy-any-machine-learning-model-serverless-in-aws/).\n\n* A **server endpoint will** remain running and can be used as an API to do inference.\n* A **batch transform job** will be deployed as a serverless application and will do inference on new data dropped in AWS S3. The moment new data is put on an S3 Bucket, a lambda function is triggered. The lambda will put the meta data of the fresh data on an AWS SQS Queue. A container will be deployed in AWS ECS, finish the jobs on the SQS Queue and finally will shut down.\n\n## Requirements\n* Make sure you've got the [aws cli installed](https://aws.amazon.com/cli/).\n* Set your `aws_access_key_id` and `aws_secret_access_key` in _~/.aws/credentials_\n* Install [Docker](https://docs.docker.com/install/)\n* Install [Serverless](https://serverless.com/framework/docs/providers/aws/guide/installation/)\n* Add extra requirements for you model in _requirements.txt_\n* If you want to do local (unit) tests, install the _requirements.txt_ to your virtual/ Anaconda environment\n\n## Your custom model and data preparation\nThis template is just a setup and needs your own model to run. When the container is spun up, the container will fetch data from the SQS Queue. You probably want to modify the data before calling the model for inference. In the current template the model as well as the scaler are Scikit-Learn objects, that need to be loaded from S3 once the container is running. The model can thus be updated by updating the files in S3.\n\n### Model artifacts\nIn _src/model/\\_\\_init\\_\\_.py_ The model, scalers and numpy arrays with mean values (in order to replace missing values) are loaded into memory. Finally the **ModelWrap** object also defines a **predict** method that is called during inference. This file must be adapted to match your needs. Currently the template only accepts ***.parquet files**. If you want to accept more file types, you'll need to adapt _src/web/router.py_ (the **transformation** function)\n\n``` python\nclass ModelWrap:\n    def __init__(self):\n        self._model = None\n        self._scaler = None\n        self._mean = None\n        self._columns = None\n\n    @property\n    def model(self):\n        \"\"\"Get the model object for this instance, loading it if it's not already loaded.\"\"\"\n        if self._model is None:\n            f = open_s3_file(app.config['BUCKET'], app.config['MODEL_PKL'])\n            self._model = pickle.load(f)\n        return self._model\n\n    @property\n    def scaler(self):\n        if self._scaler is None:\n            f = open_s3_file(app.config['BUCKET'], app.config['SCALER_PKL'])\n            self._scaler = pickle.load(f)\n        return self._scaler\n\n    @property\n    def mean(self):\n        if self._mean is None:\n            f = open_s3_file(app.config['BUCKET'], app.config['MEAN_PKL'])\n            self._mean = pickle.load(f)\n        return self._mean\n\n    @property\n    def columns(self):\n        if self._columns is None:\n            f = open_s3_file(app.config['BUCKET'], app.config['COLUMNS_PKL'])\n            self._columns = pickle.load(f)\n        return self._columns\n\n    def predict(self, x):\n        \"\"\"For the input, do the predictions and return them.\n\n        Args:\n            input (a pandas dataframe): The data on which to do the predictions. There will be\n                one prediction per row in the dataframe\"\"\"\n        id = x.iloc[:, 0]\n        x = x.iloc[:, 1:]\n        p = self.model.predict_proba(x)[:, 1]\n        return pd.DataFrame({'id': id, 'activation': p})\n```\n\n### Data preparation\nChange the **prepare_data** function in _src/model/data.py_ so that you don't have any missing values, your outliers are removed etc.\nThe following files can be modified for your specific model/ data needs.\n\n## Settings File\nAll the settings for the Docker image (such as the S3 URI's for the model and scaler) and all the settings for the Serverless application are defined the _serverless/batch-transform/serverless.yml_ under the **custom** key.\n\n``` yaml\n# Custom are the variables for this template.\ncustom:\n  flask:\n    # These setting will be exposed for the model in the docker image\n    BUCKET: \u003cbucket\u003e containing the model artifacts.\n    MODEL_PKL: \u003ckey\u003e for the above defined bucket\n    SCALER_PKL: \u003ckey\u003e for the above defined bucket\n    MEAN_PKL: \u003ckey\u003e for the above defined bucket\n    COLUMNS_PKL: \u003ckey\u003e for the above defined bucket\n    TEST_DATA: \u003ckey\u003e for the above defined bucket\n    AWS_REGION: eu-west-1\n\n  # Docker image that will be deployed\n  image: \u003crepository url\u003e\n  desiredTaskCount: 1\n\n  # Settings for the naming of new AWS resources\n  prefix: \u003cstr\u003e Resources made in AWS will have this prefix\n  usecase: \u003cstr\u003e Resoures made in AWS will have this name\n  model: \u003cstr\u003e Name of the model. Name will be given to the Resources in AWS\n\n  # Bucket \u0026 key to where the results are written\n  outputBucket: \u003cbucket\u003e\n  outputKey: \u003ckey\u003e\n\n  # Bucket that will be generated for this stack. New data should be deployed here.\n  bucket: ${self:custom.prefix}-${self:custom.usecase}-new-data\n\n  # File type that should trigger the Lambda\n  triggerExtension: .parquet\n\n  # Subnet and security group names in which the AWS Task should run.\n  subnet:  \u003csubnet name\u003e\n  securityGroup: \u003csecurity group name\u003e\n\n  # ARN of the Role that will be assigned to the Task. It needs SQS, S3 and ECS access\n  ecsTaskExecutionRole: \u003carn of role with the needed permissions\u003e\n```\n\n## Local Tests\nMake sure you've installed the _requirements.txt_, modified your code to work with your model and added your model to S3 along with some test data. If needed modify some of the unit tests in _src/tests.py_.\n\n`$ cd src \u0026\u0026 python -m unittest tests.py`\n\n## Deploy\nIf your tests worked out fine you'll need to deploy the docker image to an AWS ECR repository.\n\n`$ .build_and_push.sh \u003ctag\u003e`\n\nwill build your Dockerfile and push the ECR. If the tag doesn't yet exist, a new repository will be made. Copy the URI of the repository and add that the settings file in  _serverless/batch-transform/serverless.yml_ under **custom.image**\n\nFinally you can deploy the whole Serverless stack with:\n\n`$ cd serverless/batch-transform \u0026\u0026 sls deploy`\n\nThis command has created an AWS Cloudformation stack.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fritchie46%2Fserverless-model-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fritchie46%2Fserverless-model-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fritchie46%2Fserverless-model-aws/lists"}