{"id":24617649,"url":"https://github.com/pacphi/docker-terraform-and-jenkins","last_synced_at":"2026-04-13T14:31:40.273Z","repository":{"id":53482136,"uuid":"333318536","full_name":"pacphi/docker-terraform-and-jenkins","owner":"pacphi","description":"Fun with Docker, Terraform and Jenkins.","archived":false,"fork":false,"pushed_at":"2021-03-29T12:20:56.000Z","size":1387,"stargazers_count":0,"open_issues_count":2,"forks_count":2,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-18T21:34:18.272Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pacphi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-27T05:52:56.000Z","updated_at":"2021-03-29T12:20:58.000Z","dependencies_parsed_at":"2022-09-09T12:41:11.703Z","dependency_job_id":null,"html_url":"https://github.com/pacphi/docker-terraform-and-jenkins","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pacphi/docker-terraform-and-jenkins","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pacphi%2Fdocker-terraform-and-jenkins","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pacphi%2Fdocker-terraform-and-jenkins/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pacphi%2Fdocker-terraform-and-jenkins/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pacphi%2Fdocker-terraform-and-jenkins/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pacphi","download_url":"https://codeload.github.com/pacphi/docker-terraform-and-jenkins/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pacphi%2Fdocker-terraform-and-jenkins/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31757477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-13T13:27:56.013Z","status":"ssl_error","status_checked_at":"2026-04-13T13:21:23.512Z","response_time":93,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-24T23:39:46.502Z","updated_at":"2026-04-13T14:31:40.254Z","avatar_url":"https://github.com/pacphi.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# docker-terraform-and-jenkins\n\nFun with Docker, Terraform, Jenkins and Artifactory.\n\n![Screenshot of create-gke-cluster pipeline](create-gke-cluster.png)\n## Prerequisites\n\n* AWS\n  * [IAM Account](https://docs.aws.amazon.com/IAM/latest/UserGuide/introduction.html)\n  * Permissions to create [S3 buckets](https://docs.aws.amazon.com/AmazonS3/latest/userguide/creating-bucket.html)\n\n* Google Cloud\n  * [Service Account](https://cloud.google.com/iam/docs/creating-managing-service-accounts)\n  * 2x [VMs](https://cloud.google.com/compute/docs/quickstart-linux)\n    * 1 VM with Jenkins and Docker\n    * 1 VM with Artifactory\n  * Permissions to create [Storage buckets](https://cloud.google.com/storage/docs/creating-buckets)\n\n* Jenkins\n  * [Installation](https://www.cloudbooklet.com/how-to-install-jenkins-on-ubuntu-20-04-with-nginx-and-ssl/)\n  * Add plugins (transitive dependencies not included for brevity)\n    * [Amazon Elastic Container Service (ECS) / Fargate plugin](https://plugins.jenkins.io/amazon-ecs)\n    * [Artifactory Plugin](https://plugins.jenkins.io/artifactory)\n    * [Build Name and Description Setter](https://plugins.jenkins.io/build-name-setter)\n    * [Build Timeout](https://plugins.jenkins.io/build-timeout)\n    * [Command Agent Launcher Plugin](https://plugins.jenkins.io/command-launcher)\n    * [Configuration as Code Plugin](https://plugins.jenkins.io/configuration-as-code)\n    * [Docker Pipeline](https://plugins.jenkins.io/docker-workflow)\n    * [Docker plugin](https://plugins.jenkins.io/docker-plugin)\n    * [Docker Slaves Plugin](https://plugins.jenkins.io/docker-slaves)\n    * [docker-build-step](https://plugins.jenkins.io/docker-build-step)\n    * [Email Extension Plugin](https://plugins.jenkins.io/email-ext)\n    * [Embeddable Status Plugin](https://plugins.jenkins.io/embeddable-build-status)\n    * [Git Parameter Plug-In](https://plugins.jenkins.io/git-parameter)\n    * [GitHub Branch Source Plugin](https://plugins.jenkins.io/github-branch-source)\n    * [Google OAuth Credentials plugin](https://plugins.jenkins.io/google-oauth-plugin)\n    * [Green Balls](https://plugins.jenkins.io/greenballs)\n    * [JAXB Plugin](https://plugins.jenkins.io/jaxb)\n    * [LDAP Plugin](https://plugins.jenkins.io/ldap)\n    * [Matrix Authorization Strategy Plugin](https://plugins.jenkins.io/matrix-auth)\n    * [Oracle SE Development Kit Installer Plugin](https://plugins.jenkins.io/jdk-tool)\n    * [OWASP Markup Formatter Plugin](https://plugins.jenkins.io/antisamy-markup-formatter)\n    * [PAM Authentication Plugin](https://plugins.jenkins.io/pam-auth)\n    * [Pipeline](https://plugins.jenkins.io/workflow-aggregator)\n    * [Pipeline Utility Steps](https://plugins.jenkins.io/pipeline-utility-steps)\n    * [Pipeline: GitHub Groovy Libraries](https://plugins.jenkins.io/pipeline-github-lib)\n    * [S3 publisher plugin](https://plugins.jenkins.io/s3)\n    * [SSH Agent Plugin](https://plugins.jenkins.io/ssh-agent)\n    * [SSH Server](https://plugins.jenkins.io/sshd)\n    * [Terraform Plugin](https://plugins.jenkins.io/terraform)\n    * [Timestamper](https://plugins.jenkins.io/timestamper)\n    * [Windows Azure Storage plugin](https://plugins.jenkins.io/windows-azure-storage)\n    * [Workspace Cleanup Plugin](https://plugins.jenkins.io/ws-cleanup)\n  \n* Docker\n  * [Installation](https://linuxize.com/post/how-to-install-and-use-docker-on-ubuntu-20-04/)\n  * Images\n    * [cloud-sdk](https://cloud.google.com/sdk/docs/downloads-docker)\n    * [terraform](https://hub.docker.com/r/hashicorp/terraform/)\n\n* Artifactory\n  * [Installation](https://computingforgeeks.com/configure-jfrog-artifactory-behind-nginx-reverse-proxy-letsencrypt/)\n  * Configure a [local Generic repo](https://www.jfrog.com/confluence/display/JFROG/Repository+Management#RepositoryManagement-LocalRepositories) named `terraform-state`\n\n* CLIs\n  * [aws](https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html)\n  * GCP [cloud-sdk](https://cloud.google.com/sdk/docs/install)\n  * [terraform](https://learn.hashicorp.com/tutorials/terraform/install-cli)\n\n## Setup\n\nWe'll be managing Terraform *variables* and *secrets* in version-managed and encrypted cloud storage.  Terraform *state* will be managed in one of two ways, either within Artifactory or cloud storage.  For GCP we target Artifactory, while for AWS we target S3. (Terraform supports a number of [backends](https://www.terraform.io/docs/language/settings/backends/index.html) for state management).\n\n### Initialize buckets\n\n**on AWS**\n\n```bash\n./bin/initialize-s3-buckets.sh {suffix}\n```\n\u003e Replace `{suffix}` with a unique string.\n\n**on GCP**\n\n```bash\n./bin/initialize-gs-buckets.sh {location} {suffix}\n```\n\u003e Replace `{location}` above with a [region](https://cloud.google.com/about/locations) (e.g., `us-west1`).  Also replace `{suffix}` with a unique string.\n\n\n## Working with Terraform modules\n\nAll Terraform modules employed by Jenkins pipeline definitions are found in the [terraform](terraform) directory.  Each module constitutes a unit of capability that will ultimately be assembled and then invoked by a chained list of pipeline jobs.  You'll implement modules in separate subdirectories.  Sample configuration should be maintained alongside implementation. \n\nLet's look at an example\n\n\u003e Assumes you have a Google Cloud Platform administrator account and an existing project\n\n```bash\ncd terraform/clusters/gke\n```\n\nCopy the sample [terraform.tfvars.sample](terraform/clusters/gke/terraform.tfvars.sample) to `terraform.tfvars`. (Amend the value for each key in the new file as required and make sure that the end of this file contains a single newline).\n\nCopy the sample [backend.tf.sample](terraform/clusters/gke/backend.tf.sample) to `backend.tf`. (Amend the value for each key in the new file as required).\n\u003e It's unfortunate that we can't use variables, see this [issue](https://github.com/hashicorp/terraform/issues/13022).\n\nGo! (Effectively smoke-testing).\n\n```bash\nterraform init\nterraform plan -out terraform.plan\nterraform apply -auto-approve terraform.plan\nterraform output kubeconfig_contents \u003e ~/.kube/config\n```\n\nTeardown!\n\n```bash\nterraform destroy -auto-approve\nrm -Rf .terraform/ .terraform.lock.hcl terraform.log terraform.plan terraform.tfstate\n```\n\n## Jenkins pipeline workflow\n\nOnce you have authored and smoke-tested a module you'll want to invoke it from a Jenkins pipeline job.  Each module consumes variables and stores state in a backend provider.  Variables may contain sensitive configuration, and in some instances, configuration is owned or defined by different parties.  Therefore, it makes sense to divide configuration supplied by a *user* and an *operator*.  Users will author and maintain configuration in a Git repository.  Operators will author and maintain sensitive configuration in either Artifactory or a version-managed, encrypted cloud storage bucket.\n\nFor examples of *user-supplied* configuration visit the [user](https://github.com/pacphi/docker-terraform-and-jenkins/tree/user) branch of this Git repository.\n\nAnd here's an example of *operator-supplied* configuration\n\n```yaml\ncluster:\n  api:\n    url: \"api.pks.hamster.zoolabs.me\"\n    skip_ssl_validation: true\n    username: \"tanzu-gitops\"\n    password: \"why-would-i-share-this-with-you\"\n\nterraform:\n  backend:\n    type: \"s3\"\n    bucket: \"terraform-state-21665ca\"\n    region: \"us-west-2\"\n  buckets:\n    vars:\n      name: \"terraform-vars-21665ca\"\n    secrets:\n      name: \"terraform-secrets-21665ca\"\n```\n\nAs you implement and onboard additional modules that will participate in an overall pipeline chain, some thought and care should be placed on where configuration is defined and who is responsible for maintaining it.  Is it the *user* or the *operator*?\n\n*User-initiated* pipeline definitions (e.g., see [ci/tkgi/user-request/create/Jenkinsfile](https://github.com/pacphi/docker-terraform-and-jenkins/blob/main/ci/tkgi/user-request/create/ecs/Jenkinsfile) and [ci/tkgi/user-request/destroy/Jenkinsfile](https://github.com/pacphi/docker-terraform-and-jenkins/blob/main/ci/tkgi/user-request/destroy/ecs/Jenkinsfile)) are the places where both the *user* and *operator* configuration are consumed.  What happens when you execute the job?  Well, the configuration is merged and two (or more) files are manufactured: `terraform.tfvars` and `backend.tf`.  A `terraform.tfvars` file is manufactured for each downstream module.  We're basically seeding configuration for downstream pipeline jobs to consume.\n\n### Upload operator manifest\n\n**on AWS**\n\n```bash\n./bin/upload-operator-manifest-to-s3-buckets.sh {path-to-manifest} {s3-bucket-name} {environment} {manifest-filename}\n```\n\nSample\n\n```bash\n./bin/upload-operator-manifest-to-s3-bucket.sh ../dev terraform-secrets-21665ca dev tkgi-cr.v1.yaml\n```\n\n**on GCP**\n\n```bash\n./bin/upload-operator-manifest-to-gs-buckets.sh {path-to-manifest} {gs-bucket-name} {environment} {manifest-filename}\n```\n\nSample\n\n```bash\n./bin/upload-operator-manifest-to-gs-bucket.sh ../test terraform-secrets-076f328 test gke-cr.v1.yaml\n```\n### Service accounts \n\n**on GCP**\n\nYou'll need to upload a copy of the [service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys#iam-service-account-keys-create-gcloud) (in json format).  The file should be named `gcp-service-account.json`.\n\n```bash\ngsutil cp gcp-service-account.json gs://sa-credentials-{suffix}/gcp-service-account.json\n```\n\u003e Replace `{suffix}` above with same string you defined when you created the bucket\n\n\n### Author Jenkinsfile \n\nYou'll need to create some [pipelines](https://www.jenkins.io/doc/book/pipeline/jenkinsfile/) now.\n\nFor inspiration have a look in the [ci/gke](ci/gke) and [ci/tkgi](ci/tkgi) directories.\n\n### Upload Jenkinsfile\n\nLogin to the Jenkins instance via your favorite browser.\n\nClick on *New Item*.\n\n*Enter an item name* then select *Pipeline* and click the *OK* button.\n\nScroll down to the *Pipeline* section.  Under the *Definition* sub-section make sure that *Pipeline script* is selected from the drop-down selector, then \ncopy-and-paste the contents of a *Jenkinsfile* into the *Script* block.  Click the *Save* button to complete uploading the *Jenkinsfile*.\n\nYou might choose to organize related pipelines into a view.\n\n![Screenshot of Jenkins view](jenkins-view.png)\n\n\u003e The collection of `-v2` suffixed pipelines you see in the screenshot above are based upon the `Jenkinsfile` pipelines found in the `ecs` sub-directories underneath the `ci/tkgi` sub-directory tree.\n\n## Additional reading\n\n* [Running Jenkins jobs in AWS ECS with slave agents](https://tomgregory.com/jenkins-jobs-in-aws-ecs-with-slave-agents/)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpacphi%2Fdocker-terraform-and-jenkins","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpacphi%2Fdocker-terraform-and-jenkins","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpacphi%2Fdocker-terraform-and-jenkins/lists"}