{"id":15222449,"url":"https://github.com/rsachdeva/illuminatingdeposits-gcp-trigger","last_synced_at":"2026-02-11T13:02:24.713Z","repository":{"id":65804611,"uuid":"592365876","full_name":"rsachdeva/illuminatingdeposits-gcp-trigger","owner":"rsachdeva","description":"Terraform usage in the context of Google Cloud Platform GCP based Trigger of Resources applied to Cloud Functions. Both resource creation and destruction is through Terraform.","archived":false,"fork":false,"pushed_at":"2023-07-14T23:47:07.000Z","size":907,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-08T12:02:37.662Z","etag":null,"topics":["bigquery","bigquery-table","cloud-events","functions-framework","gcp","go","golang","golangci-lint","google-cloud","google-cloud-function-pubsub-trigger","google-cloud-functions","google-cloud-pubsub","google-cloud-sdk","google-cloud-storage","google-cloud-terraform","sendgrid","terraform"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsachdeva.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-01-23T15:23:45.000Z","updated_at":"2024-09-09T14:09:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"d99db550-f55e-4c55-9abd-a19d720a527d","html_url":"https://github.com/rsachdeva/illuminatingdeposits-gcp-trigger","commit_stats":{"total_commits":9,"total_committers":1,"mean_commits":9.0,"dds":0.0,"last_synced_commit":"e575cc6a19b23f82111bfd33847f17ecc33187e9"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rsachdeva/illuminatingdeposits-gcp-trigger","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-gcp-trigger","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-gcp-trigger/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-gcp-trigger/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-gcp-trigger/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsachdeva","download_url":"https://codeload.github.com/rsachdeva/illuminatingdeposits-gcp-trigger/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsachdeva%2Filluminatingdeposits-gcp-trigger/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29333155,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-11T12:42:24.625Z","status":"ssl_error","status_checked_at":"2026-02-11T12:41:23.344Z","response_time":97,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","bigquery-table","cloud-events","functions-framework","gcp","go","golang","golangci-lint","google-cloud","google-cloud-function-pubsub-trigger","google-cloud-functions","google-cloud-pubsub","google-cloud-sdk","google-cloud-storage","google-cloud-terraform","sendgrid","terraform"],"created_at":"2024-09-28T15:12:04.364Z","updated_at":"2026-02-11T13:02:24.697Z","avatar_url":"https://github.com/rsachdeva.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Illuminating Deposits - Google Cloud Platform (GCP) trigger of Resources applied to Golang (Go) Google Cloud Functions (GCF 2nd Generation)\n\n### What is **Delta** in the context of this project?\n\n**Delta** is the approx. value of 30 days Interest for a deposit. So if a deposit is for 10 years the Delta\nis still calculated for 30 days Interest using the Interest for 10 years to get an approx. value of 30 days Interest.\nHence, the variation in interest money in 30 days is called Delta. This value Delta is used to compare as a mechanism to\nlet you know which deposits are performing better for you.\n\n(Development is WIP)\n\nIlluminating Deposits Project Logo:\n\n![Illuminating Deposits Project Logo](logo.png \"Illuminating Deposits Project Logo\")\n\n## Overall google cloud architecture system design:\n![google cloud architecture system design](GoogleCloudArchitectureSystemDiagram.svg \"google cloud architecture system design\")\n\nCreated from link:\n\n[Google Cloud Architecture](https://googlecloudcheatsheet.withgoogle.com/architecture)\n\nGREEN here means these initial cloud functions have been implemented, development is in progress.\nRED Text And GREEN Stroke boxes means partially completed in progress\nThere is only 1 box in RED + GEEN and that is the box for the Google Cloud Function (GCF) gcf_analytics.\n\n**---------------------------**\n## Cloud Function Deploy: gcf_upload\ngcf_upload google cloud function is triggered by HTTP request and submits HTTP POST request body to\ngoogle cloud storage bucket for further processing.\n\nTerraform is used to deploy gcf_upload to google cloud.\nThe service account used for deployment needs keys to be generated and stored in the deploy/terraform folder\nwith the file being renamed to keys.json.\nThe gcf_upload is written in Go and uses google cloud storage and google cloud functions.\n\n### Make steps for gcf_upload deployment in cloud:\nAdd alias tf=terraform in .zshrc or equivalent\nSteps start from root of project folder\n1. `cd gcf_upload`\n2. `make init`\n3. `make apply`\n   In the end on successful creation you will get something like:\n   `google_cloudfunctions2_function.illuminating_gcf_upload: Creation complete`\n4. Dependent Google cloud functions should be destroyed first. So if you are also working on\n   gcf_interestcal also for example (see below) then see destruction in gcf_interestcal.\n   If you are only playing around with this Google cloud function,\n   after you are done using this function and no longer need for any processing, `make destroy`\n   In the end on successful destruction you will get something like:\n   `google_storage_bucket.illuminating_gcf_upload_bucket: Destruction complete`\n   Also, See Related [Multiple Cloud Functions](#multiple-cloud-functions)\n\n#### Testing gcf_upload in cloud:\n1. `make cloud-incorrect-json`\n2. `make cloud-not-successful-http-request-accounttypemissing`\n3. `make cloud-not-successful-http-request-banknamemissing`\n4. `make cloud-successful-http-request`\n\n### Make steps for gcf_upload deployment locally:\nSteps start from root of project folder\n1. `cd gcf_upload`\n2. `make gcf-local`\n\n#### Testing gcf_upload locally:\n1. `cd gcf_upload`\n2. `make local-incorrect-json`\n3. `make local-not-successful-http-request-accounttypemissing`\n4. `make local-not-successful-http-request-banknamemissing`\n5. `make local-successful-http-request`\n   Here this sends Cloud Storage event to a CloudEvent function running at localhost:8080 using curl request.\n\n### Seeing logs in console on the web for a cloud function:\n1. Go to Log tab of Google Cloud Function\n2. Click Log Explorer\n3. At the bottom Click Edit Time to see logs for the last 15 minutes\n   (even this can be edited to say last 10 minutes)\n   Per the retention period these logs delete after 30 days. There is no charge for the 30-day period.\n   ([https://cloud.google.com/logging#section-7](https://storage.cloud.google.com/illuminating_upload_json_bucket/inputrequest.json))\n4. Click Refresh button to see the logs\n   Also, See Related [Multiple Cloud Functions](#multiple-cloud-functions)\n\nTo see logs using gcloud command line:\n* `gcloud config set project illuminatingdeposits-gcp`\n* `gcloud auth login `\n* `sudo pip3 install grpcio`\n* `export CLOUDSDK_PYTHON_SITEPACKAGES=1`\n* `gcloud alpha logging tail --format=\"default(timestamp,text_payload)\"`\n\n**---------------------------**\n\n## Cloud Function Deploy: gcf_interestcal\ngcf_interestcal google cloud function is triggered by upload to illuminating_upload_json_bucket_input by gcf_upload cloud\nfunction and starts calculating Delta for each bank account including overall for all deposits. It then also\nuploads the calculated  Deltas with deposit details to illuminating_upload_json_bucket_output google cloud storage bucket\nfor further processing.\n\nDepends on gcf_upload cloud function deployed and having run successfully. See System Diagram for more details.\n\nTerraform is used to deploy gcf_interestcsal to google cloud.\nThe service account used for deployment needs keys to be generated and stored in the deploy/terraform folder\nwith the file being renamed to keys.json.\nThe gcf_interest is written in Go and uses google cloud storage and google cloud functions.\n\n### Make steps for gcf_interestcal deployment in cloud:\nAdd alias tf=terraform in .zshrc or equivalent\nSteps start from root of project folder\n1. `cd gcf_interestcal`\n2. `make init`\n3. `make apply`\n   In the end on successful creation you will get something like:\n   `google_cloudfunctions2_function.illuminating_gcf_interestcal: Creation complete`\n4. After you are done using this function and no longer need for any processing, `make destroy`\n   In the end on successful destruction you will get something like:\n   `google_storage_bucket.illuminating_gcf_interestcal_bucket: Destruction complete`\n   Also, See Related [Multiple Cloud Functions](#multiple-cloud-functions)\n\n#### Testing gcf_interestcal in cloud:\n`make cloud-successful-storage-upload`\nThis copies the file locally to cloud storage\n\n### Make steps for gcf_interestcal deployment locally:\nSteps start from root of project folder\n1. `cd gcf_interestcal`\n2. `make gcf-local`\n\n#### Testing gcf_interestcal locally:\n`make local-successful-send-event-to-cloud-event-function`\nThis only triggers the function locally and does not copy the file to cloud storage.\nHere this sends Cloud Storage event to a CloudEvent function running at localhost:8080 using curl request.\n\n**---------------------------**\n\n\n## Cloud Function Deploy: gcf_analytics\ngcf_analytics google cloud function is triggered by upload to illuminating_upload_json_bucket_output by gcf_interestcal cloud\nfunction and starts appending the calculations to BigQuery `delta_calculations` table in `gcfdeltaanalytics` table.\nScope: Analytics triggered by a Submit of Interest Request only supported currently.\nCurrently,\n* This appends a row in BigQuery for Interest Request Submitted. There is also env variable for query only mode to bypass this fpr development mode.\n* Based on the interest request submitted, this does query on BigQuery for delta calculations overall for last 5 requests to\n  compare the delta calculations. This result is placed in Analytics struct\n* Then it also Publishes as JSON the analytics message to PubSub.\n\nDepends on gcf_interestcal cloud function deployed and having run successfully. See System Diagram for more details.\n\n### Make steps for gcf_analytics deployment in cloud:\nAdd alias tf=terraform in .zshrc or equivalent\nSteps start from root of project folder\n1. `cd gcf_analytics`\n2. `make init`\n3. `make apply`\n4.  In the end on successful creation you will get something like:\n    `google_cloudfunctions2_function.illuminating_gcf_analytics: Creation complete`\n5. After you are done using this function and no longer need for any processing, `make destroy`\n   In the end on successful destruction you will get something like:\n   `google_storage_bucket.illuminating_gcf_analytics_bucket: Destruction complete`\n   Also, See Related [Multiple Cloud Functions](#multiple-cloud-functions)\n\n#### Testing gcf_analytics in cloud:\n`make cloud-successful-storage-upload`\nThis copies the file locally to cloud storage\n\n### Make steps for gcf_analytics deployment locally:\nSteps start from root of project folder\n1. `cd gcf_analytics`\n2. `make gcf-local`\n\n#### Testing gcf_analytics locally:\n`make local-successful-send-event-to-cloud-event-function`\nThis only triggers the function locally and does not copy the file to cloud storage.\nYou can send an event to CloudEvent function locally with already existing file in cloud storage\nHere this sends Cloud Storage event to a CloudEvent function running at localhost:8080 using curl request.\n\n\n\n## Cloud Function Deploy: gcf_notify\nThis Google cloud function is triggered by message published to pubsub topic `deltaanalyticstopic` by\ngcf_analytics cloud function. Uses google cloud secret manager grabbing sendgrid api key to send email message\ndecoded from base64 encoding per JSON message schema received from pubsub.\nOnly sends from verified sender.\nSends to env email address for now, in future version will be able to\nsend to email address specified in request\n\nDepends on gcf_analytics cloud function deployed and having run successfully. See System Diagram for more details.\n\n### Make steps for gcf_notify deployment in cloud:\nAdd alias tf=terraform in .zshrc or equivalent\nSteps start from root of project folder\n1. `cd gcf_notify`\n2. `make init`\n3. `make apply`\n4.  In the end on successful creation you will get something like:\n    `google_cloudfunctions2_function.illuminating_gcf_notify: Creation complete`\n5. After you are done using this function and no longer need for any processing, `make destroy`\n   In the end on successful destruction you will get something like:\n   `google_storage_bucket.illuminating_gcf_notify_bucket: Destruction complete`\n   Also, See Related [Multiple Cloud Functions](#multiple-cloud-functions)\n\n#### Testing gcf_notify in cloud:\n`make cloud-pubsub-message-successful`\nThis publishes a message to pubsub topic\n\n### Make steps for gcf_notify deployment locally:\nSteps start from root of project folder\n1. `cd gcf_notify`\n2. `make gcf-local`\n\n#### Testing gcf_notify locally:\n`make local-successful-send-event-to-cloud-event-function`\nThis triggers the function locally by sending message to deltaanalyticstopic.\nHere this sends Cloud Pub/Sub event to a CloudEvent function running at localhost:8080 using curl request.\n\n**---------------------------**\n\n## Multiple Cloud Functions\n\n### Multiple Cloud Functions Deploy: gcf_upload and gcf_interestcal and gcf_analytics\nAdd alias tf=terraform in .zshrc or equivalent\nSteps start from root of project folder at `illuminatingdeposits-gcp-trigger`\n\n1. `make init`\n2. `make apply`\n3. `make destroy`\n\n#### Integration Testing gcf_upload, gcf_interestcal, gcf_analytics and gcf_notify in cloud:\nSince everything gets triggered with gcf_upload, we will test gcf_upload , gcf_interestcal, gcf_analytics and gcf_notify together using gcf_upload http request submission:\n1. `cd gcf_upload`\n2. `make cloud-successful-http-request`\n3. To view logs in Google Cloud [Log Explorer](https://console.cloud.google.com/logs/) for all cloud functions change query tab text to:\n   `(resource.type = \"cloud_run_revision\"\n   resource.labels.service_name =\"illuminating-gcf-upload\" OR resource.labels.service_name = \"illuminating-gcf-interestcal\" OR resource.labels.service_name = \"illuminating-gcf-analytics\" OR resource.labels.service_name = \"illuminating-gcf-notify\"\n   resource.labels.location = \"us-central1\")\n   severity\u003e=DEFAULT`\n4. Click Run Query\n5. Adjust time to see logs from last 10 minutes or whatever as needed\n6. You should see in log email sent. This happens after the analytics are calculated and sent to pubsub topic.\n7. you should receive an email set in env variable `RECEIVER_EMAIL` in gcf_notify cloud function with the custom message.\n\n**---------------------------**\n\n\n# Version\nv0.5.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsachdeva%2Filluminatingdeposits-gcp-trigger","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsachdeva%2Filluminatingdeposits-gcp-trigger","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsachdeva%2Filluminatingdeposits-gcp-trigger/lists"}