{"id":16813160,"url":"https://github.com/jmhbnz/gitlab-gcp-deployment","last_synced_at":"2026-04-13T10:32:13.750Z","repository":{"id":107980193,"uuid":"326836206","full_name":"jmhbnz/gitlab-gcp-deployment","owner":"jmhbnz","description":"Using org-mode literate programming techniques to deploy Gitlab on a Google Cloud Platform virtual machine via Docker.","archived":false,"fork":false,"pushed_at":"2021-02-24T04:46:00.000Z","size":87,"stargazers_count":0,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-02-22T15:48:25.967Z","etag":null,"topics":["bash","docker","docker-compose","emacs","gcp","org-mode"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jmhbnz.png","metadata":{"files":{"readme":"README.org","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-04T23:50:22.000Z","updated_at":"2021-02-24T04:46:02.000Z","dependencies_parsed_at":"2023-07-13T11:30:46.362Z","dependency_job_id":null,"html_url":"https://github.com/jmhbnz/gitlab-gcp-deployment","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jmhbnz/gitlab-gcp-deployment","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmhbnz%2Fgitlab-gcp-deployment","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmhbnz%2Fgitlab-gcp-deployment/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmhbnz%2Fgitlab-gcp-deployment/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmhbnz%2Fgitlab-gcp-deployment/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jmhbnz","download_url":"https://codeload.github.com/jmhbnz/gitlab-gcp-deployment/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jmhbnz%2Fgitlab-gcp-deployment/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31749030,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T09:16:15.125Z","status":"ssl_error","status_checked_at":"2026-04-13T09:16:05.023Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","docker","docker-compose","emacs","gcp","org-mode"],"created_at":"2024-10-13T10:25:01.682Z","updated_at":"2026-04-13T10:32:13.730Z","avatar_url":"https://github.com/jmhbnz.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"#+TITLE: Gitlab GCP Deployment\n#+AUTHOR: James Blair\n#+EMAIL: mail@jamesblair.net\n#+DATE: 5th January 2021\n\n[[https://github.com/jmhbnz/gitlab-gcp-deployment/workflows/CI/badge.svg]]\n\nThis org file is intended to capture and automate the end to end workflow to deploy an instance of [[https://gitlab.com][Gitlab]] on [[https://console.cloud.google.com][Google Cloud Platform]].\n\nWe'll use shell blocks inside this file which can be executed with [[https://orgmode.org/worg/org-contrib/babel/][Babel]]. Additionally we want to explore tangling these source code blocks to shell script files within this document so that the scripts can then be executed by a continous delivery pipeline.\n\n*Notes:*\n 1. To interact with this org file we're using the [[https://github.com/humacs/humacs][Humacs]] distribution of [[https://www.gnu.org/software/emacs/][Emacs]].\n 1. This workflow has only been tested on the ~Ubuntu 20.04~ linux distribution, via [[https://ubuntu.com/wsl][WSL 2]].\n\n* Step 1 - Ensure GCP SDK is installed\n\nTo automate our interactions with Google Cloud Platform we'll use the [[https://cloud.google.com/sdk/docs/install#deb][GCP SDK]] which provides us with a number of command line tools to interact with the platform, such as ~gcloud~, ~gsutil~ and ~kubectl~.\n\nTangle the shell block below to a shell script by pressing *, b t* in emacs command mode:\n\n#+NAME: Install google cloud sdk\n#+BEGIN_SRC bash :shebang #!/bin/bash :tangle 1-install-utilities.sh\n# Add the Cloud SDK distribution URI as a package source\necho \"deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main\" | sudo tee /etc/apt/sources.list.d/google-cloud-sdk.list\n\n# Make sure apt-transport-https is installed\nsudo apt-get install -y apt-transport-https ca-certificates gnupg\n\n# Import the Google Cloud public key\ncurl https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -\n\n# Update and install the SDK\nsudo apt-get update \u0026\u0026 sudo apt-get install -y google-cloud-sdk\n\n# Install GitHub CLI\nsudo apt-key adv --keyserver keyserver.ubuntu.com --recv-key C99B11DEB97541F0\nsudo apt-add-repository https://cli.github.com/packages\nsudo apt-get update \u0026\u0026 sudo apt-get install gh\n#+END_SRC\n\n\n* Step 2 - Configure cloud resources\n\nWith GCP SDK now installed we need to authenticate, create a project and then create a virtual machine instance that we will install Gitlab into later in the workflow.\n\nFirst up is authentication so our GCP SDK installation can carry out actions in a given account and project. This part of the process is currently a manual step as the authentication process includes some interactive steps.\n\nIn future we could automate this process as part of a continous delivery pipeline using a GCP service account with permission to create virtual machine instances.\n\n#+NAME: Authenticate with google cloud platform\n#+BEGIN_SRC bash :shebang #!/bin/bash :tangle no\ngcloud auth login\n#+END_SRC\n\nNext step is we need to authenticate with GitHub CLI. This part is the process is currently manual.\nWe required GitHub CLI to store secrets to automate CI/CD pipeline for gcloud access.\n#+NAME: Authenticate with GitHub CLI\n#+begin_src bash\ngh auth login\n#+end_src\n\nOnce we have authenticated we can create a project and then create a new virtual machine instance within that project.\n\nFirstly let's create a new project, a project is the logical boundary all our cloud resources for this deployment will live within. To be able to deploy resources we also need to enable billing.\n\nTangle the shell block below to a shell script by pressing *, b t* in emacs command mode:\n\n#+NAME: Create a new google cloud project\n#+begin_src bash :shebang #!/bin/bash :tangle 2-configure-gcp-project.sh\n# Create a project id based on date\nexport gcp_project_id=\"gitlab-gcp-ci\"\n\n# Create new project using a random project id\ngcloud projects create $gcp_project_id\n\n# Ensure billing is enabled for the project\nexport gcp_billing_account=$(gcloud alpha billing accounts list --limit=1 --format='value(name.basename())')\ngcloud alpha billing projects link $gcp_project_id --billing-account $gcp_billing_account\n\n# Make sure the project is set active\ngcloud config set project $gcp_project_id\n\n# Create service accounts\ngcloud iam service-accounts create $gcp_project_id-sa \\\n  --description=\"GitHub actions deploy account\" \\\n  --display-name=\"GitHub-Action-Deploy\"\n\n# Assign role\ngcloud projects add-iam-policy-binding $gcp_project_id \\\n  --member=serviceAccount:$gcp_project_id-sa@$gcp_project_id.iam.gserviceaccount.com \\\n  --role=roles/compute.instanceAdmin.v1\n\ngcloud projects add-iam-policy-binding $gcp_project_id \\\n  --member=serviceAccount:$gcp_project_id-sa@$gcp_project_id.iam.gserviceaccount.com \\\n  --role=roles/iam.serviceAccountUser\n\ngcloud iam service-accounts keys create key.json \\\n  --iam-account $gcp_project_id-sa@$gcp_project_id.iam.gserviceaccount.com\n#+end_src\n\n\n* Step 3 - Configure GitHub secrets...\n\nOnce new service account is created and key.json file is created, We will have to upload these details into GitHub and automate the end to end CI/CD workflow.\n\nTangle the shell block below to a shell script by pressing *, b t* in emacs command mode:\n\n#+begin_src bash :shebang #!/bin/bash :tangle 3-create-github-secrets.sh\necho $gcp_project_id | gh secret set GCP_PROJECT_ID\ncat key.json | jq '.client_email' -r | gh secret set GCP_EMAIL\ncat key.json | gh secret set GCP_CREDENTIALS\n#+end_src\n\n\n* Step 4 - Create virtual machine\n\nOnce we have a project we can create a new virtual machine. To create a virtual machine we need to ensure compute engine apis are enabled.\n\nTangle the shell block below to a shell script by pressing *, b t* in emacs command mode:\n\n#+begin_src bash :shebang #!/bin/bash :tangle 4-create-virtual-machine.sh\n# Set default project\ngcloud config set project \"gitlab-gcp-ci\"\n\n# Ensure compute engine apis are enabled in the project\ngcloud services enable compute.googleapis.com\n\n# Create name for virtual machine based on date\nexport gcp_machine_name=\"gitlab-gcp-\"$(date +\"%s\")\n\n# Create the new machine\ngcloud compute instances create $gcp_machine_name --zone australia-southeast1-a\n#+end_src\n\n\n* Step 5 - Install docker on virtual machine\n\nNext up we need to install [[https://docker.com][Docker]] on the newly created virtual machine so that we can then deploy Gitlab as a container.\n\nBy default the virtual machine operating system for the vm we created on GCP is [[https://debian.org][Debian]]. There are instructions for installing Docker on a debian machine [[https://docs.docker.com/engine/install/debian/#install-using-the-repository][here]].\n\n#+begin_src bash :shebang #!/bin/bash :tangle 5-install-docker.sh\n# Set default zone\ngcloud config set compute/zone australia-southeast1-a\n\n# Retrieve the vm name\nexport gcp_machine_name=$(gcloud compute instances list --limit=1 --format='value(name.basename())')\n\n# Connect to the machine using ssh\ngcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- \"curl -fsSL https://get.docker.com -o get-docker.sh \u0026\u0026 sudo sh get-docker.sh\"\n\n# Install docker compose\ngcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- \"sudo curl -L https://github.com/docker/compose/releases/download/1.27.4/docker-compose-$(uname -s)-$(uname -m) -o /usr/local/bin/docker-compose\"\n\ngcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- \"sudo chmod +x /usr/local/bin/docker-compose\"\n#+end_src\n\n\n* Step 6 - Install gitlab via docker\n\nNext step to create docker compose file for gitlab\n\n#+begin_src yaml :tangle docker-compose.yaml\n   web:\n     image: 'gitlab/gitlab-ce:latest'\n     restart: always\n     hostname: 'gitlab.example.com'\n     environment:\n       GITLAB_OMNIBUS_CONFIG: |\n         external_url 'https://gitlab.example.com:3200'\n     # Add any other gitlab.rb configuration here, each on its own line\n     ports:\n       - '3200:3200'\n       - '443:443'\n       - '2222:22'\n     volumes:\n       - '$GITLAB_HOME/config:/etc/gitlab'\n       - '$GITLAB_HOME/logs:/var/log/gitlab'\n       - '$GITLAB_HOME/data:/var/opt/gitlab'\n\n#+end_src\n\nNext up we need to tranfer file to gcloud vm and install the gitlab via docker composer\n\n#+begin_src bash :shebang #!/bin/bash :tangle 6-install-gitlab-via-composer.sh\n# Retrieve the vm name\nexport gcp_machine_name=$(gcloud compute instances list --limit=1 --format='value(name.basename())')\n\n# Copy file to vm\ngcloud compute scp docker-compose.yaml $gcp_machine_name:/home/$USER --ssh-key-file ~/.ssh/$USER --strict-host-key-checking=no\n\n# Install gitlab with docker compose\ngcloud compute ssh $gcp_machine_name --ssh-key-file ~/.ssh/$USER -- 'sudo docker-compose up -d'\n#+end_src\n\n\n* Step 7 - Teardown cloud resources\n\nThe Google Cloud Platform resources created by this process come at a cost, so it's important we have an easy way to teardown those resources as soon as we're finished with them!\n\nThe script below will delete any projects containing ~gitlab~ in the name along with any compute instances running in those projects.\n\nTangle the shell block below to a shell script by pressing *, b t* in emacs command mode:\n\n#+begin_src bash :shebang #!/bin/bash :tangle 7-teardown-cloud-resources.sh\n# Iterate over any matching projects\nfor project in $(gcloud projects list | awk '{ print $1 }' | grep gitlab); do\n\n  # Iterate over any instances in the project\n  for instance in $(gcloud compute instances list --project $project --format=\"value(name)\"); do\n\n    # Delete the instance\n    gcloud compute instances delete --quiet $instance --zone australia-southeast1-a --project $project\n\n  done\n\n  # Delete the project as well\n  gcloud projects delete $project --quiet\n\ndone\n#+end_src\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmhbnz%2Fgitlab-gcp-deployment","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjmhbnz%2Fgitlab-gcp-deployment","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjmhbnz%2Fgitlab-gcp-deployment/lists"}