{"id":22428490,"url":"https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference","last_synced_at":"2026-05-02T09:32:47.102Z","repository":{"id":261195064,"uuid":"871679534","full_name":"kwame-mintah/gcp-cloud-run-function-model-inference","owner":"kwame-mintah","description":"A cloud run function to invoke a prediction against a machine learning model that has been trained outside of a cloud provider.","archived":false,"fork":false,"pushed_at":"2025-01-19T22:36:42.000Z","size":137,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-01T11:43:46.185Z","etag":null,"topics":["cloud-functions","fastapi","gcp","google-cloud-platform","model-inference"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kwame-mintah.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-12T16:29:13.000Z","updated_at":"2025-01-19T22:36:44.000Z","dependencies_parsed_at":"2024-12-21T02:32:08.635Z","dependency_job_id":"d731edda-cf8a-4b75-9691-3b3af76e3f8d","html_url":"https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference","commit_stats":null,"previous_names":["kwame-mintah/gcp-cloud-function-model-inference","kwame-mintah/gcp-cloud-run-function-model-inference"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fgcp-cloud-run-function-model-inference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fgcp-cloud-run-function-model-inference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fgcp-cloud-run-function-model-inference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwame-mintah%2Fgcp-cloud-run-function-model-inference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kwame-mintah","download_url":"https://codeload.github.com/kwame-mintah/gcp-cloud-run-function-model-inference/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245798538,"owners_count":20673901,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud-functions","fastapi","gcp","google-cloud-platform","model-inference"],"created_at":"2024-12-05T20:15:00.039Z","updated_at":"2026-05-02T09:32:47.063Z","avatar_url":"https://github.com/kwame-mintah.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Google Cloud Platform (GCP) Cloud Run Function Model Inference\n\n[![🚧 Bump version](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/run-version-bump.yml/badge.svg)](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/run-version-bump.yml)\n[![🚀 Push Docker image to GCP Artifact Registry](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-docker-build-and-push.yml/badge.svg)](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-docker-build-and-push.yml)\n[![🛸 GCP Cloud Run Deploy](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-cloud-run-deploy.yml/badge.svg)](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-cloud-run-deploy.yml)\n[![🛰️ GCP Cloud Run Delete](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-cloud-run-delete.yml/badge.svg)](https://github.com/kwame-mintah/gcp-cloud-run-function-model-inference/actions/workflows/gcp-cloud-run-delete.yml)\n\nA cloud function to invoke a prediction against a machine learning model that has been trained outside\na cloud provider, using tools like [MLFlow](https://mlflow.org/). This repository will not contain the model artifact output,\nbut the code for the cloud function.\n\nFastAPI will be used for the cloud function as it offers many features e.g. authentication, body validation etc.\nand overall easy to use and maintain. Note all GCP resources are created within this repository, some resources are created\nvia Terraform in this repository [terraform-gcp-model-serving](https://github.com/kwame-mintah/terraform-gcp-model-serving).\n\n# Disclaimer\n\nThere are many options of serving a model, this project aims to demonstrate one. Each machine learning (ML) project is different\nother options approaches that could be considered and related to this project is the following:\n\n1. Include the model artifact in the docker image, instead of downloading from Google Cloud Platform (GCP) bucket,\n2. Utilizing GCP [cloud run volumes](https://cloud.google.com/sdk/gcloud/reference/run/deploy#--add-volume) and storing the model artifact within a volume and can load the model in memory.\n3. Instead of having to upload the model artifact each time, the default artifact root to the [GCP bucket](https://mlflow.org/docs/latest/tracking/artifacts-stores.html#google-cloud-storage)[^1]\n\n# Architecture\n\n![proposed-model-inference-architecture](./docs/drawio/cloud-function-model-inference-overview.png)\n\n1. Machine learning model is trained outside GCP and model artifact output created,\n2. User makes a request to a HTTP endpoint for a prediction,\n3. Model artifact is stored within a bucket, when function is invoked -- model is downloaded,\n4. Prediction is output via a HTTP response.\n\n# Training using MLflow\n\nAs with all machine learning projects, your milage may vary (YMMV). This project will re-use existing data set for\n[wine quality](https://archive.ics.uci.edu/dataset/186/wine+quality) using [mlflow-example](https://github.com/mlflow/mlflow-example)\nfor demonstration. The provided `docker-compose.yml` file will create the necessary resources needed to train locally.\nThe following steps below will start the services:\n\n1. Start mlflow server, postgres and minio:\n\n```shell\ndocker compose up -d --build\n```\n\n2. Access MLflow UI with http://localhost:5001\n3. Access MinIO UI with http://localhost:9000*\n4. Next start a training job within the `mlflow_server` container using the [cli](https://mlflow.org/docs/latest/cli.html#mlflow-run):\n\n```shell\ndocker exec mlflow_server mlflow run https://github.com/mlflow/mlflow-example.git -P alpha=0.42\n```\n\n5. Once training has started, you should be able to view the run under 'Experiments' tab in MLflow UI\n6. You can download the `model.pkl` using the UI under 'Artifacts' tab, but should be available locally under `/mlartifacts/`\n7. Copy the model artifact `model.pkl` to the root directory of the project ready to be used locally in FastAPI\n\n`*` Login with credentials used in `docker-compose.yml`.\n\n\u003e [!NOTE]\n\u003e Because the MLflow is a custom docker image, passing `--build` arg will cause the docker image to be re-built each time\n\u003e which is helpful when amending the `.mlflow/requirements.txt`. A re-build is not needed each time, if there is no changes\n\u003e being made to the file and `---build` can be omitted from the command.\n\n## Running FastAPI\n\nThe following environment variables need to be set before attempting to run the application:\n\n| Environment variable name             | Description                                                                 | Default | Required |\n|---------------------------------------|-----------------------------------------------------------------------------|---------|----------|\n| GCP_MLFLOW_MODEL_ARTIFACT_BUCKET_NAME | The GCP bucket name, where the model artifact has been uploaded             | N/A     | Yes      |\n| USE_LOCAL_FILE_PATH_MODEL             | Use the model artifact found locally, rather than fetching from GCP bucket. | False   | No       |\n\nThe following steps below will start the FastAPI service locally:\n\n1. Install python packages used for the service:\n\n   ```shell\n   pip install -r requirements.txt\n   ```\n\n2. Run the FastAPI server, which will start on port 8000:\n\n   ```shell\n   python main.py\n   ```\n\n   Endpoint documentation is available on: http://127.0.0.1:8000/docs\n\n# Prediction with FastAPI\n\nThe application exposes  a single `/predict/*` endpoint, which allows the user to send a list of various quantitative\nfeatures needed to predict the wine quality. An example payload for predicting wine quality for one wine can be found below:\n\n```json\n[\n  {\n    \"alcohol\": 12.8,\n    \"chlorides\": 0.029,\n    \"citric acid\": 0.48,\n    \"density\": 0.98,\n    \"fixed acidity\": 6.2,\n    \"free sulfur dioxide\": 29,\n    \"pH\": 3.33,\n    \"residual sugar\": 1.2,\n    \"sulphates\": 0.39,\n    \"total sulfur dioxide\": 75,\n    \"volatile acidity\": 0.66\n  }\n]\n```\n\nWhich will return an HTTP 200 Successful and a score e.g. `[3.6182495833379846]`.\n\n## GitHub Action (CI/CD)\n\nThe GitHub Action \"🚀 Push Docker image to GCP Artifact Registry\" will check out the repository and push a docker image\nto the chosen GCP Artifact Registry using [setup-gcloud](https://github.com/google-github-actions/setup-gcloud/tree/v2.1.2) action.\nThe following repository secrets need to be set:\n\n| Secret                                    | Description                                                            |\n|-------------------------------------------|------------------------------------------------------------------------|\n| GCP_GITHUB_SERVICE_ACCOUNT_KEY            | The json private key for the GitHub service account                    |\n| GCP_PREDICTION_SERVICE_ACCOUNT_KEY_BASE64 | The json private key base64 encoded for the prediction service account |\n\n\u003e [!IMPORTANT]\n\u003e The `GCP_PREDICTION_SERVICE_ACCOUNT_KEY_BASE64` must be base64 encoded this can be done with the following command e.g.\n\u003e `base64 -i \u003cservice_account\u003e.json -o prediction_service_account.base64`. During the workflow run this will be decoded\n\u003e and used as part of the docker image build.\n\nAdditionally, the following variables need to be set:\n\n| Secret                       | Description                           |\n|------------------------------|---------------------------------------|\n| GCP_PROJECT_ID               | The GCP Project ID                    |\n| GCP_REGION                   | The region that project is in         |\n| GCP_REGISTRY_REPOSITORY_NAME | The artifact registry repository name |\n\nSecondly, GitHub Action \"🛸 GCP Cloud Run Deploy\" will check out the repository and deploy the cloud run function utilizing\nthe same GitHub action mentioned above. The following repository variable needs to be set:\n\n| Secret                                | Description                                                     |\n|---------------------------------------|-----------------------------------------------------------------|\n| GCP_MLFLOW_MODEL_ARTIFACT_BUCKET_NAME | The GCP bucket name, where the model artifact has been uploaded |\n\n\nLastly, GitHub Action \"🛰️ GCP Cloud Run Delete\" will check out the repository and delete the cloud run function utilizing\nthe same GitHub action mentioned above and repository variables.\n\n# References\n\n1. [How to serve deep learning models using TensorFlow 2.0 with Cloud Functions](https://cloud.google.com/blog/products/ai-machine-learning/how-to-serve-deep-learning-models-using-tensorflow-2-0-with-cloud-functions) by Rustem Feyzkhanov\n\n[^1]: Within the `minio` can run the following command `mc alias set gcs https://storage.googleapis.com \u003cYOUR-ACCESS-KEY\u003e \u003cYOUR-SECRET-KEY\u003e`\n      and check that you can list the contents of the bucket e.g. `mc ls gcs/\u003cYOUR-BUCKET-NAME\u003e`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwame-mintah%2Fgcp-cloud-run-function-model-inference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwame-mintah%2Fgcp-cloud-run-function-model-inference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwame-mintah%2Fgcp-cloud-run-function-model-inference/lists"}