{"id":24175826,"url":"https://github.com/iht/beam-cloud-build-terraform","last_synced_at":"2026-03-06T17:04:21.672Z","repository":{"id":223457845,"uuid":"760334298","full_name":"iht/beam-cloud-build-terraform","owner":"iht","description":"The scripts in this repo will build the Apache Beam Java SDK packages, using Cloud Build and Artifact Registry, for a personal Beam fork.","archived":false,"fork":false,"pushed_at":"2025-09-07T11:57:30.000Z","size":71,"stargazers_count":0,"open_issues_count":3,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-09-07T13:24:31.633Z","etag":null,"topics":["apache-beam","artifact-registry","cloud-build","google-cloud"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/iht.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-20T08:20:14.000Z","updated_at":"2025-02-11T17:39:23.000Z","dependencies_parsed_at":"2024-02-20T11:27:48.163Z","dependency_job_id":"aa09b02b-b87d-4ce5-9fad-43414c1a13d8","html_url":"https://github.com/iht/beam-cloud-build-terraform","commit_stats":null,"previous_names":["iht/beam-cloud-build-terraform"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/iht/beam-cloud-build-terraform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iht%2Fbeam-cloud-build-terraform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iht%2Fbeam-cloud-build-terraform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iht%2Fbeam-cloud-build-terraform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iht%2Fbeam-cloud-build-terraform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/iht","download_url":"https://codeload.github.com/iht/beam-cloud-build-terraform/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/iht%2Fbeam-cloud-build-terraform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30186780,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-06T14:42:24.748Z","status":"ssl_error","status_checked_at":"2026-03-06T14:42:14.925Z","response_time":250,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-beam","artifact-registry","cloud-build","google-cloud"],"created_at":"2025-01-13T02:33:09.633Z","updated_at":"2026-03-06T17:04:21.645Z","avatar_url":"https://github.com/iht.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Custom Apache Beam builds in Google Cloud\n\nThe scripts in this repo will build the Apache Beam Java SDK packages, using Cloud Build and\nArtifact Registry, for a personal Beam fork.\n\nThis repo contains Terraform code to setup Cloud Build and Artifact Registry, to build and publish\ncustom Apache Beam packages, for testing and development purposes, in a private package repository.\nFor instance, if you want to contribute a pull request to Apache Beam, but you would like first to\nuse your code with your own pipelines, this makes it easier to publish and use those custom packages\nwith your pipelines, before submitting the pull request.\n\nThe Terraform code assumes that you have your repo in Github.\n\n## Pre-requirements\n\nTo use this custom builds, you need the following:\n\n- A project in Google Cloud\n- A fork of the [Apache Beam repository](https://github.com/apache/beam)\n- A fork of the [Beam Cloud Builder repo](https://github.com/iht/beam-cloud-builder)\n\nYou can use the Cloud Shell to run this Terraform code. Alternatively, if you prefer to use your\nlocal computer, you will need to install and configure the following dependencies:\n\n- Google Cloud SDK: \u003chttps://cloud.google.com/sdk/docs/install-sdk\u003e\n- Terraform \u003e= 1.5.1: \u003chttps://developer.hashicorp.com/terraform/install\u003e\n\n## Create the build triggers and package repositories\n\n### Step 1\n\nLink your Github forks (`beam` and `beam-cloud-builder`) with Cloud Build. This is a manual step\nthat cannot be done using Terraform. Follow the instructions given at:\n\n- \u003chttps://cloud.google.com/build/docs/automating-builds/github/connect-repo-github\u003e\n\n**Use 1st generation repository linking with Cloud Build**. The Terraform code will not work with\n2nd generation repository linking.\n\n### Step 2\n\nCreate a file named `terraform.vars` with the following content:\n\n```hcl\nbeam_repo = \"beam\"\ncontainer_repo = \"beam-cloud-builder\"\ngithub_owner = \"\u003cYOUR GITHUB USERNAME\u003e\"\nproject_id = \"\u003cYOUR GOOGLE CLOUD PROJECT ID\u003e\"\nregion = \"\u003cA CLOUD BUILD REGION ID\u003e\"\n```\n\nFor the Cloud Build region id, choose one from the list of supported locations.\n\n- \u003chttps://cloud.google.com/build/docs/locations\u003e\n\nFor instance, to run your builds in the Madrid region, use `europe-southwest1`.\n\n### Step 3\n\nRun the Terraform code. In the shell (e.g. Cloud Shell), run the following commands:\n\n```shell\nterraform init\nterraform apply\n```\n\nReview the changes that the code will make in your Google Cloud project and snswer `yes` when\nprompted.\n\nMake sure you are authenticated in Google Cloud with an user or service\naccount that has enough permissions to enable the Cloud Build and Artifact Registry APIs, and to\ncreate the repositories and triggers.\n\nWhen this step is completed, go to the list of triggers and choose the build region:\n\n- \u003chttps://console.cloud.google.com/cloud-build/triggers\u003e\n\nYou should see a list of triggers like the following:\n\n![List of triggers in Cloud Build](./imgs/triggers.png)\n\n### Step 4\n\nRun the Beam Cloud Builder trigger to publish the container image that is used to build Beam. Use\nthe branch `main` from the image fork. You can run the trigger manually, or use the following\ncommand in the shell (e.g. Cloud Shell):\n\n```shell\ngcloud builds triggers run --region=\u003cYOUR CLOUD BUILD REGION\u003e --branch=main beam-cloud-builder-trigger\n```\n\nThat will publish an image in the builder Artifact Registry Repo. You can check the images in the\nrepo `beam-build-docker` in Artifact Registry:\n\n- \u003chttps://console.cloud.google.com/artifacts\u003e\n\nThis steps is a pre-requirement for the rest of the builds to work correctly.\n\n## How are builds triggered?\n\nThe builds will listen to changes in any branch of any of the forks. Any time you push changes to\nany of the branches of your repo, the lates (new) commit of each branch will built.\n\nThe packages are published with the following version number pattern:\n\n- `X.XX.X-\u003cSHORT COMMIT ID\u003e-SNAPSHOT`\n\nFor instance:\n\n- `2.55.0-8a74a1c2-SNAPSHOT`\n\nSo every time you are writing to a branch in your fork, with the idea of later sending a pull\nrequest, you will be able to easily identify what packages correspond exactly to your changes\n(branch and short commit id).\n\nYou can also try to run some of the triggers manually, using the branch `master` (the main branch\nin the Apache Beam repo), to make sure everything is working correctly.\n\n## How can I use the published packages?\n\nIf you have `gcloud` configured in the same project, you can some commands to get the settings for\nGradle and Maven. If you are in the Cloud Shell, `gcloud` is already configured for you.\n\nRun the following commands first:\n\n```shell\ngcloud config set artifacts/repository\ngcloud config set artifacts/location \u003cYOUR CLOUD BUILD REGION\u003e\n```\n\nFor **Gradle settings**, run and copy the output in your `build.gradle` file:\n\n```shell\ngcloud artifacts print-settings gradle\n```\n\nFor **Maven settings**, run and copy the output in your `pom.xml` file:\n\n```shell\ngcloud artifacts print-settings mvn\n```\n\nThere are more detailed instructions in the Artifact Registry documentation:\n\n- \u003chttps://cloud.google.com/artifact-registry/docs/java\u003e\n\n## How to undo all the changes?\n\nIf you want to remove all the changes done by these scripts, you can run the following command:\n\n```shell\nterraform destroy\n```\n\n**WARNING: this will remove all the packages and containers** published in the Artifact Registry\nrepositories. You will not be able to recover those packages and containers.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiht%2Fbeam-cloud-build-terraform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fiht%2Fbeam-cloud-build-terraform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fiht%2Fbeam-cloud-build-terraform/lists"}