{"id":19030556,"url":"https://github.com/jeffbryner/gcp-project-pipeline","last_synced_at":"2026-02-26T22:03:14.690Z","repository":{"id":139750618,"uuid":"507125341","full_name":"jeffbryner/gcp-project-pipeline","owner":"jeffbryner","description":"Goldilocks project to allow a simple way to bootstrap a gcp project with a CICD pipeline to terraform subsequent project bits and bobbles","archived":false,"fork":false,"pushed_at":"2022-06-24T19:46:20.000Z","size":25,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-21T20:16:07.575Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jeffbryner.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-06-24T19:40:14.000Z","updated_at":"2022-06-24T19:46:24.000Z","dependencies_parsed_at":null,"dependency_job_id":"43266497-5fa3-4ab5-9e88-381dafa39c6f","html_url":"https://github.com/jeffbryner/gcp-project-pipeline","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jeffbryner/gcp-project-pipeline","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffbryner%2Fgcp-project-pipeline","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffbryner%2Fgcp-project-pipeline/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffbryner%2Fgcp-project-pipeline/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffbryner%2Fgcp-project-pipeline/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jeffbryner","download_url":"https://codeload.github.com/jeffbryner/gcp-project-pipeline/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jeffbryner%2Fgcp-project-pipeline/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29874489,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-26T21:05:00.265Z","status":"ssl_error","status_checked_at":"2026-02-26T20:57:13.669Z","response_time":89,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T21:18:32.640Z","updated_at":"2026-02-26T22:03:14.661Z","avatar_url":"https://github.com/jeffbryner.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# GCP-Project-Pipeline\n\nA goldilocks effort for bootstraping a GCP project with it's own ci/cd pipeline:\n\nInspired by these great repos:\n\n  - https://github.com/GoogleCloudPlatform/healthcare-data-protection-suite\n  - https://github.com/terraform-google-modules/terraform-example-foundation\n\n## Why?\nThe healthcare data protection suite is great at making a simple project that gets you a functional ci/cd pipeline for terraforming GCP all using GCP resources. However, it's templated terraform eminating from a golang engine. This project cuts to the chase using just terraform.\n\nThe terraform example foundation is about building a complete GCP org, not an individual project and can be overkill. It also doesn't setup source control/triggers for the cicd pipeline itself, which is odd.\n\nThis is meant to simply satisfy the urge to dream up a project, give it a name and location in the org tree, terraform apply and then immediately begin using the ci/cd pipeline to build the remaining infrastructure.\n\n\n## Continuous integration (CI) and continuous deployment (CD)\n\nThe CI and CD pipelines use\n[Google Cloud Build](https://cloud.google.com/cloud-build) and\n[Cloud Build triggers](https://cloud.google.com/cloud-build/docs/automating-builds/create-manage-triggers)\nto detect changes in a cloud source repo, trigger builds, and implement terraform changes.\n\n## Concept of operations\nThe desired end state is a GCP project with a cloud source repo and cloud build triggers that terraform plan/apply on code commits. The build triggers will run terraform against a list of directories you choose (cicd by default) so you can use it to build the pipeline, infrastructure, groups, compute, databases, iam, etc as separate concerns as you see fit.\n\nThe `_managed_dirs` list in the `triggers.tf` file in the cicd directory sets the directories that will be managed by this cicd pipeline.\n\n## Setup\nYou will need to be able to create a project with billing in the appropriate place in your particular org structure. First you'll run terraform locallly to initialize the project and the pipeline. After the project is created, we will transfer terraform state to the cloud bucket and from then on you can use git commits to trigger terraform changes without any local resources or permissions.\n\n1. Clone this repo\n\n2. Change directory (cd) to the cicd directory and edit the terraform.tfvars to match your GCP organization.\n\n3. Run the following commands in the cicd directory to enable the necessary APIs,\n   grant the Cloud Build service account the necessary permissions, and create\n   Cloud Build triggers and the terraform state bucket:\n\n    ```shell\n    terraform init\n    terraform apply\n    ```\n4. Get the name of the terraform state bucket from the terraform output\n\n    ```shell\n    terraform output\n    ```\n  and copy backend.tf.example as backend.tf with the proper bucket name.\n\n    ```terraform\n        terraform {\n      backend \"gcs\" {\n        bucket = \"UPDATE_ME_WITH_OUTPUT_OF_INITIAL_INIT\"\n        prefix = \"cicd\"\n      }\n    }\n    ```\n\n  Note that if you create other directories for other terraform concerns, you should duplicate this backend.tf file in those directories with a different prefix so your state bucket matches your directory layout.\n\n5. Now terraform can transfer state from your local environment into GCP. From the cicd directory:\n    ```shell\n    terraform init -force-copy\n    ```\n\n6. Follow the instructions at https://source.cloud.google.com/\u003cproject name\u003e/\u003crepository name\u003e to then push your code (from the parent directory of cicd, i.e. not the cicd directory) into your new CICD pipeline. Basically:\n\n    ```shell\n    git init\n    gcloud init \u0026\u0026 git config --global credential.https://source.developers.google.com.helper gcloud.sh\n    git remote add google  https://source.developers.google.com/p/\u003cproject name\u003e/r/\u003crepository name\u003e\n    git checkout -b main\n    git add cicd/configs/* cicd/backend.tf cicd/main.tf cicd/outputs.tf cicd/terraform.tfvars cicd/triggers.tf cicd/variables.tf\n    git commit -m 'first push after bootstrap'\n    git push --all google\n\n7. After the repo and pipeline is established you should be able to view the build triggers and history by visiting:\nhttps://console.cloud.google.com/cloud-build/dashboard?project=\u003cproject id here\u003e\n\n\n## CICD Container/Container Creation\n\nThe Docker container used for CICD executions are inspired by those built and maintained by the\nCloud Foundation Toolkit (CFT) team.\n\n\nDocumentations and source can be found [here](https://github.com/GoogleCloudPlatform/cloud-foundation-toolkit/tree/master/infra/build/developer-tools-light). Images can be found [here](https://console.cloud.google.com/gcr/images/cloud-foundation-cicd/GLOBAL/cft/developer-tools-light).\n\nThis version of the container includes necessary dependencies (e.g. bash, terraform, gcloud, python, pip, pipenv) to validate and deploy Terraform configs and is based on hashicorp's native terraform and alpine linux.\n\nTo build the container, cd to the container directory and issue the command\n\n```bash\ngcloud builds submit\n```\nWhich will kick off a build using the cloudbuild.yaml and Dockerfile in the container directory, creating a 'cloudbuilder' container ( gcr.io/${PROJECT_ID}/cloudbuilder )in your project that is used by the triggers.\n\n\n## Features\n\n### Event-triggered builds\n\nTwo presubmit and one postsubmit triggers are created by default.\n\n* \\[Presubmit\\] `tf-validate`: Perform Terraform format and syntax check.\n  * It does not access Terraform remote state.\n* \\[Presubmit\\] `tf-plan`: Generate speculative plans to show a set of\n    potential changes if the pending config changes are deployed.\n  * It accesses Terraform remote state but does not lock it.\n  * This also performs a non-blocking check for resource deletions. These\n        are worth reviewing, as deletions are potentially destructive.\n* \\[Postsubmit\\] `tf-apply`: Apply the terraform configs that are checked into\n    the config source repo.\n  * It accesses Terraform remote state and locks it.\n  * This trigger is only applicable post-submit.\n  * When this trigger is set in the Terraform engine config, the Cloud Build\n        service account is given broader permissions to be able to make changes\n        to the infrastructure.\n\nEvery new push to the Pull Request at the configured branches automatically\ntriggers presubmit runs.\n\nThe postsubmit Cloud Build job automatically starts after a Pull Ruquest is\nsubmitted to a configured branch. To view the result of the Cloud Build run, go\nto [Build history](https://console.cloud.google.com/cloud-build/builds) and look\nfor your commit to view the Cloud Build job triggered by your merged commit.\n\nThe `build_viewers` members can view detailed log output.\n\nThe triggers all use a [helper runner script](./cicd/configs/run.sh) to perform\nactions. The `DIRS` var within the script lists the directories that are managed by the triggers and the order they are\nrun.\n\n### Deletion check allowlist\n\nThe deletion check run as part of the `tf-plan` trigger optionally accepts an\nallowlist of resources to ignore, using\n[grep extended regex patterns](https://en.wikipedia.org/wiki/Regular_expression#POSIX_extended)\nmatched against the Terraform resource **address** from the plan.\n\nTo configure an allowlist:\n\n1. Create a file `tf-deletion-allowlist.txt` in the `cicd/configs/` directory.\n2. Add patterns to it, one per line.\n\nExample:\n\n```text\nnetwork\n^module.cloudsql.module.safer_mysql.google_sql_database.default$\ngoogle_sql_user.db_users\\[\"user-creds\"\\]\n```\n\nEach line allows, respectively:\n\n1. Any resource whose address contains the string \"network\".\n2. A specific resource within a module.\n3. A specific resource with a generated name, i.e. from `for_each` or `count`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffbryner%2Fgcp-project-pipeline","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjeffbryner%2Fgcp-project-pipeline","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjeffbryner%2Fgcp-project-pipeline/lists"}