{"id":19792683,"url":"https://github.com/making/cfcr-aws","last_synced_at":"2026-05-16T18:36:05.930Z","repository":{"id":136908467,"uuid":"134370559","full_name":"making/cfcr-aws","owner":"making","description":"How to deploy Cloud Foundry Container Runtime (formerly known as Kubo/Kubernete on BOSH) on AWS","archived":false,"fork":false,"pushed_at":"2018-10-03T06:43:00.000Z","size":1397,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2026-01-21T16:02:51.739Z","etag":null,"topics":["aws","bosh","cfcr","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/making.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-22T06:29:51.000Z","updated_at":"2023-03-22T18:24:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"d45c125d-0c3c-4d0f-9be0-b2bad6c8a40f","html_url":"https://github.com/making/cfcr-aws","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/making/cfcr-aws","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/making%2Fcfcr-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/making%2Fcfcr-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/making%2Fcfcr-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/making%2Fcfcr-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/making","download_url":"https://codeload.github.com/making/cfcr-aws/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/making%2Fcfcr-aws/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33114493,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T04:41:52.686Z","status":"ssl_error","status_checked_at":"2026-05-16T04:41:52.009Z","response_time":115,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","bosh","cfcr","kubernetes"],"created_at":"2024-11-12T07:07:41.395Z","updated_at":"2026-05-16T18:36:05.914Z","avatar_url":"https://github.com/making.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"## How to deploy Cloud Foundry Container Runtime (Kubernetes on BOSH) on AWS\n\n### Deploy CFCR\n\nI have customized [the official way](https://docs-cfcr.cfapps.io/installing/) to install CFCR on AWS so that it would be more bosh-friendly.\nAlso changed Terraform template to get rid of manual operations by hands.\n\n#### Pave the AWS environment with Terraform\n\nFirst we will pave the AWS environment required for BOSH and Kubernete. We will use Terraform here.\n\nObtain [customized Terraform template]((https://github.com/making/cfcr-aws)).\n\n```bash\ngit clone https://github.com/making/cfcr-aws.git\ncd cfcr-aws/terraform\n```\n\nPrepare `terraform.tfvars`. `access_key` and `secret_key` are for an IAM user to run Terraform who has the `AdministratorAccess` Role.\n\n```bash\ncat \u003c\u003cEOF \u003e terraform.tfvars\nprefix             = \"changeme\"\naccess_key         = \"changeme\"\nsecret_key         = \"changeme\"\nregion             = \"ap-northeast-1\"\navailability_zones = [\"ap-northeast-1a\",\"ap-northeast-1c\",\"ap-northeast-1d\"]\nvpc_cidr           = \"10.0.0.0/16\"\nnat_instance_type  = \"t2.nano\"\nEOF\n```\n\nExecute Terraform with the following command.\n\n```\nterraform init\nterraform plan -out plan\nterraform apply plan\n```\n\nThe environment as shown in the following figure should be made.\n\n![image](https://user-images.githubusercontent.com/106908/42409133-b6fc430a-8210-11e8-9970-4adcec6a4bf6.png)\n\n#### Login to Bastion server\n\nNext we will provision BOSH on a paved environment, but we will do the work on the Bastion server.\n\n```bash\ncat terraform.tfstate | jq -r '.modules[0].resources[\"tls_private_key.deployer\"].primary.attributes.private_key_pem' \u003e deployer.pem\nchmod 600 deployer.pem\nexport BASTION_IP=`cat terraform.tfstate | jq -r '.modules[0].outputs[\"bosh_bastion_ip\"].value'`\n\necho \"ssh -o StrictHostKeyChecking=no -i $(pwd)/deployer.pem ubuntu@${BASTION_IP}\" \u003e ssh-bastion.sh\nchmod +x ssh-bastion.sh\n```\n\nExecute ssh login to Bastion server by the following script.\n\n```bash\n./ssh-bastion.sh\n```\n\n#### Provision BOSH\n\nProvision BOSH (BOSH Director) using [bosh-deployment](https://github.com/cloudfoundry/bosh-deployment). \nWe also have [kubo-deployment](https://github.com/cloudfoundry-incubator/kubo-deployment) and manage with git.\n\n```bash\nmkdir cfcr-manifests\ncd cfcr-manifests\ngit init\ngit submodule add https://github.com/cloudfoundry/bosh-deployment.git\ngit submodule add https://github.com/cloudfoundry-incubator/kubo-deployment.git\ncd kubo-deployment\ngit checkout v0.21.0\ncd ..\ngit add -A\ngit commit -m \"import CFCR v0.21.0\"\n```\n\nWe will manage the difference file (ops-file) of YAML in the `ops-files` directory.\n\n```bash\nmkdir -p ops-files\n```\n\nCreate an ops-file that makes the BOSH Director VM size smaller (`t2.small`).\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/director-size-aws.yml\n- type: replace\n  path: /resource_pools/name=vms/cloud_properties/instance_type\n  value: t2.small\nEOF\n```\n\nCreate a script to provisioning BOSH. Environment variables have already been set to Bastion server in Terraform.\n\n```bash\ncat \u003c\u003c'EOF' \u003e deploy-bosh.sh\n#!/bin/bash\nbosh create-env bosh-deployment/bosh.yml \\\n    -o bosh-deployment/aws/cpi.yml \\\n    -o bosh-deployment/uaa.yml \\\n    -o bosh-deployment/credhub.yml \\\n    -o bosh-deployment/jumpbox-user.yml \\\n    -o bosh-deployment/local-dns.yml \\\n    -o ops-files/director-size-aws.yml \\\n    -o kubo-deployment/configurations/generic/dns-addresses.yml \\\n    -o kubo-deployment/configurations/generic/bosh-admin-client.yml \\\n    -o kubo-deployment/manifests/ops-files/iaas/aws/bosh/tags.yml \\\n    -v director_name=bosh-aws \\\n    -v internal_cidr=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}') \\\n    -v internal_gw=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}' | sed 's|0/24|1|') \\\n    -v internal_ip=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}' | sed 's|0/24|252|') \\\n    -v access_key_id=${AWS_ACCESS_KEY_ID} \\\n    -v secret_access_key=${AWS_SECRET_ACCESS_KEY} \\\n    -v region=${region} \\\n    -v az=$(echo ${availability_zones} | awk -F ',' '{print $1}') \\\n    -v default_key_name=${default_key_name} \\\n    -v default_security_groups=[${default_security_groups}] \\\n    --var-file private_key=${HOME}/deployer.pem \\\n    -v subnet_id=$(echo ${private_subnet_ids} | awk -F ',' '{print $1}') \\\n    --vars-store=bosh-aws-creds.yml \\\n    --state bosh-aws-state.json\nEOF\nchmod +x deploy-bosh.sh\n```\n\nExecute the script to create BOSH Director.\n\n```bash\n./deploy-bosh.sh\n```\n\n![image](https://user-images.githubusercontent.com/106908/42380355-92610628-8168-11e8-9bcc-c0701a68b3b7.png)\n\nThe environment as shown in the following figure should be made.\n\n![image](https://user-images.githubusercontent.com/106908/42409141-ce8ee860-8210-11e8-99c1-e57d6f8cb552.png)\n\nIf you want to update the BOSH Director , you can do `git pull` in `bosh-deployment` directory and then re-run `./deploy-bosh`.\n\n#### BOSH Director settings\n\nMake settings to access BOSH Director and log in to BOSH Director.\n\n```bash\ncat \u003c\u003c'EOF' \u003e bosh-aws-env.sh\nexport BOSH_CLIENT=admin  \nexport BOSH_CLIENT_SECRET=$(bosh int ./bosh-aws-creds.yml --path /admin_password)\nexport BOSH_CA_CERT=$(bosh int ./bosh-aws-creds.yml --path /director_ssl/ca)\nexport BOSH_ENVIRONMENT=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}' | sed 's|0/24|252|')\nEOF\nchmod +x bosh-aws-env.sh\n```\n\nExecute the following command.\n\n\n```bash\nsource bosh-aws-env.sh\n```\n\nConfirm `bosh env` and `bosh login`.\n\n```\n$ bosh env\nUsing environment '10.0.8.252' as client 'admin'\n\nName      bosh-aws  \nUUID      7ad78602-70fa-434f-a79d-d5dda6006366  \nVersion   266.4.0 (00000000)  \nCPI       aws_cpi  \nFeatures  compiled_package_cache: disabled  \n          config_server: enabled  \n          dns: disabled  \n          snapshots: disabled  \nUser      admin  \n\nSucceeded\n```\n\n```\n$ bosh login\nSuccessfully authenticated with UAA\n\nSucceeded\n```\n\n#### Upload Stemcell\n\nUpload Stemcell which is the template image of VM created by BOSH.\n\n```\nSTEMCELL_VERSION=97.15 # latest as of writing\nbosh upload-stemcell https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-xenial-go_agent?v=${STEMCELL_VERSION}\n```\n\n#### Upate Cloud Config\n\nWe will create Cloud Config to set the IaaS environment on BOSH Director.\n\nWe use [oficial template]((https://github.com/cloudfoundry-incubator/kubo-deployment/blob/v0.21.0/configurations/aws/cloud-config.yml)) for the template of Cloud Config, \nbut because `vm_type`'s name is different from the values used in [`cfcr.yml`](https://github.com/cloudfoundry-incubator/kubo-deployment/blob/v0.21.0/manifests/cfcr.yml)\nwe create ops-file to rename ...\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/cloud-config-rename-vm-types.yml\n- type: replace\n  path: /vm_types/name=master/name\n  value: small\n- type: replace\n  path: /vm_types/name=worker/name\n  value: small-highmem\n- type: replace\n  path: /compilation/vm_type\n  value: small-highmem\nEOF\n```\n\nMake `instance_type`s smaller.\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/cloud-config-small-vm-types.yml\n- type: replace\n  path: /vm_types/name=minimal/cloud_properties/instance_type\n  value: t2.micro\n- type: replace\n  path: /vm_types/name=small/cloud_properties/instance_type\n  value: t2.micro\n- type: replace\n  path: /vm_types/name=small-highmem/cloud_properties/instance_type\n  value: t2.medium\nEOF\n```\n\nMake `vm_extensions` to attach a load balancer to the Master API .\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/cloud-config-master-lb.yml\n- type: replace\n  path: /vm_extensions?/-\n  value:\n    name: master-lb\n    cloud_properties:\n      elbs:\n      - ((master_target_pool))\nEOF\n```\n\nIf you use an ALB instead of a classic ELB,\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/cloud-config-master-lb.yml\n- type: replace\n  path: /vm_extensions?/-\n  value:\n    name: master-lb\n    cloud_properties:\n      lb_target_groups:\n      - ((master_target_pool))\nEOF\n```\n\nEnable multi-az\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/cloud-config-multi-az.yml\n- type: replace\n  path: /azs/name=z1/cloud_properties/availability_zone\n  value: ((az1_name))\n\n- type: replace\n  path: /azs/name=z2/cloud_properties/availability_zone\n  value: ((az2_name))\n\n- type: replace\n  path: /azs/name=z3/cloud_properties/availability_zone\n  value: ((az3_name))\n\n- type: replace\n  path: /networks/name=default\n  value:\n    name: default\n    subnets:\n    - az: z1\n      gateway: ((az1_gateway))\n      range: ((az1_range))\n      reserved:\n      - ((az1_gateway))/30\n      cloud_properties:\n        subnet: ((az1_subnet))\n      dns:\n      - ((dns_recursor_ip))\n    - az: z2\n      gateway: ((az2_gateway))\n      range: ((az2_range))\n      reserved:\n      - ((az2_gateway))/30\n      cloud_properties:\n        subnet: ((az2_subnet))\n      dns:\n      - ((dns_recursor_ip))\n    - az: z3\n      gateway: ((az3_gateway))\n      range: ((az3_range))\n      reserved:\n      - ((az3_gateway))/30\n      cloud_properties:\n        subnet: ((az3_subnet))\n      dns:\n      - ((dns_recursor_ip))\n    type: manual\nEOF\n```\n\nCreate a script to update Cloud Config.\n\n```bash\ncat \u003c\u003c'EOF' \u003e update-cloud-config.sh\n#!/bin/bash\nbosh update-cloud-config kubo-deployment/configurations/aws/cloud-config.yml \\\n    -o ops-files/cloud-config-rename-vm-types.yml \\\n    -o ops-files/cloud-config-small-vm-types.yml \\\n    -o ops-files/cloud-config-master-lb.yml \\\n    -o ops-files/cloud-config-multi-az.yml \\\n    -v master_iam_instance_profile=${prefix}-cfcr-master \\\n    -v worker_iam_instance_profile=${prefix}-cfcr-worker \\\n    -v az1_name=$(echo ${availability_zones} | awk -F ',' '{print $1}') \\\n    -v az1_range=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}') \\\n    -v az1_gateway=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}' | sed 's|0/24|1|') \\\n    -v az1_subnet=$(echo ${private_subnet_ids} | awk -F ',' '{print $1}') \\\n    -v az2_name=$(echo ${availability_zones} | awk -F ',' '{print $2}') \\\n    -v az2_range=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $2}') \\\n    -v az2_gateway=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $2}' | sed 's|0/24|1|') \\\n    -v az2_subnet=$(echo ${private_subnet_ids} | awk -F ',' '{print $2}') \\\n    -v az3_name=$(echo ${availability_zones} | awk -F ',' '{print $3}') \\\n    -v az3_range=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $3}') \\\n    -v az3_gateway=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $3}' | sed 's|0/24|1|') \\\n    -v az3_subnet=$(echo ${private_subnet_ids} | awk -F ',' '{print $3}') \\\n    -v dns_recursor_ip=$(echo ${private_subnet_cidr_blocks} | awk -F ',' '{print $1}' | awk -F '.' '{print $1\".\"$2\".0.2\"}') \\\n    -v access_key_id=${AWS_ACCESS_KEY_ID} \\\n    -v secret_access_key=${AWS_SECRET_ACCESS_KEY} \\\n    -v region=${region} \\\n    -v master_target_pool=${prefix}-cfcr-api\nEOF\nchmod +x update-cloud-config.sh\n```\n\nExecute the following command.\n\n```bash\n./update-cloud-config.sh\n```\n\n#### Deploy a Kubernetes cluster\n\nDeployment of Kubernetes is done based on [official manifest](https://github.com/cloudfoundry-incubator/kubo-deployment/blob/v0.21.0/manifests) with the difference applied by ops-files.\n\nCreate an ops-file to use CFCR 0.21.0.\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/kubernetes-kubo-0.21.0.yml\n- type: replace\n  path: /releases/name=kubo?\n  value:\n    name: kubo\n    version: 0.21.0\n    url: https://bosh.io/d/github.com/cloudfoundry-incubator/kubo-release?v=0.21.0\n    sha1: 24d6f17865dbf0e68cef4495c6d5bdcb46b4a132\nEOF\n```\n\nCreate an ops-file that reduces the number of instances of Worker to 1. (Please change this value if you want to increase Worker)\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/kubernetes-worker.yml\n- type: replace\n  path: /instance_groups/name=worker/instances\n  value: 1\nEOF\n```\n\nCreate an ops-file that adds `vm_extensions` to attach LB to Master and DNS name of ELB to SAN of Master's TLS certificate.\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/kubernetes-master-lb.yml\n- type: replace\n  path: /instance_groups/name=master/vm_extensions?/-\n  value: master-lb\n\n- type: replace\n  path: /variables/name=tls-kubernetes/options/alternative_names/-\n  value: ((kubernetes_master_host))\nEOF\n```\n\nAddon which we want to register additionally at the time of deployment is smanaged under the `spec` directory.\n\n```bash\nmkdir -p specs\n```\n\nCreate a spec that registers StorageClass for EBS as default.\n\n```yaml\ncat \u003c\u003cEOF \u003e specs/aws-storage-class.yml\napiVersion: storage.k8s.io/v1\nkind: StorageClass\nmetadata:\n  name: standard\n  annotations:\n    storageclass.beta.kubernetes.io/is-default-class: \"true\"\n  labels:\n    kubernetes.io/cluster-service: \"true\"\n    addonmanager.kubernetes.io/mode: EnsureExists\nprovisioner: kubernetes.io/aws-ebs\nallowVolumeExpansion: true\nparameters:\n  type: gp2\nEOF\n```\n\nCreate a script to deploy Kubrnetes. We will use single master node.\n\n```bash\ncat \u003c\u003c'EOF' \u003e deploy-kubernetes.sh\n#!/bin/bash\nbosh deploy -d cfcr kubo-deployment/manifests/cfcr.yml \\\n    -o kubo-deployment/manifests/ops-files/misc/single-master.yml \\\n    -o kubo-deployment/manifests/ops-files/addons-spec.yml \\\n    -o kubo-deployment/manifests/ops-files/iaas/aws/lb.yml \\\n    -o kubo-deployment/manifests/ops-files/iaas/aws/cloud-provider.yml \\\n    -o kubo-deployment/manifests/ops-files/use-coredns.yml \\\n    -o ops-files/kubernetes-kubo-0.21.0.yml \\\n    -o ops-files/kubernetes-worker.yml \\\n    -o ops-files/kubernetes-master-lb.yml \\\n    --var-file addons-spec=\u003c(for f in `ls specs/*.yml`;do cat $f;echo;echo \"---\";done) \\\n    -v kubernetes_cluster_tag=${kubernetes_cluster_tag} \\\n    -v kubernetes_master_host=${master_lb_ip_address} \\\n    --no-redact\nEOF\nchmod +x deploy-kubernetes.sh\n```\n\nExecute the following command to deploy. ( You need to enter `y` halfway.)\n\n```bash\n./deploy-kubernetes.sh\n```\n\nThe environment as shown in the following figure should be made.\n\n![image](https://user-images.githubusercontent.com/106908/42381708-e6535570-816c-11e8-8a00-a4773cf192ad.png)\n\nRun `bosh vms` and `bosh instances --ps` to see the VM list and process list.\n\n```\n$ bosh -d cfcr vms\nUsing environment '10.0.8.252' as client 'admin'\n\nTask 13. Done\n\nDeployment 'cfcr'\n\nInstance                                     Process State  AZ  IPs       VM CID               VM Type        Active  \nmaster/0c9bf70c-db82-482d-b38f-fd05dfe0819d  running        z1  10.0.8.4  i-0662d0d4a543d63ba  small          true  \nworker/d9808c82-0e18-4a45-87ce-c57b9874db2f  running        z1  10.0.8.5  i-0da422703dd4ecb8c  small-highmem  true  \n\n2 vms\n\nSucceeded\n```\n\n```\n$ bosh -d cfcr instances --ps\nUsing environment '10.0.8.252' as client 'admin'\n\nTask 12. Done\n\nDeployment 'cfcr'\n\nInstance                                           Process                  Process State  AZ  IPs  \napply-addons/73dd46d1-14ba-4b1b-8bd8-488f2ea3baaf  -                        -              z1  -  \nmaster/0c9bf70c-db82-482d-b38f-fd05dfe0819d        -                        running        z1  10.0.8.4  \n~                                                  bosh-dns                 running        -   -  \n~                                                  bosh-dns-healthcheck     running        -   -  \n~                                                  bosh-dns-resolvconf      running        -   -  \n~                                                  etcd                     running        -   -  \n~                                                  flanneld                 running        -   -  \n~                                                  kube-apiserver           running        -   -  \n~                                                  kube-controller-manager  running        -   -  \n~                                                  kube-scheduler           running        -   -  \nworker/d9808c82-0e18-4a45-87ce-c57b9874db2f        -                        running        z1  10.0.8.5  \n~                                                  bosh-dns                 running        -   -  \n~                                                  bosh-dns-healthcheck     running        -   -  \n~                                                  bosh-dns-resolvconf      running        -   -  \n~                                                  docker                   running        -   -  \n~                                                  flanneld                 running        -   -  \n~                                                  kube-proxy               running        -   -  \n~                                                  kubelet                  running        -   -  \n\n18 instances\n\nSucceeded\n```\n\nIf you want to updat CFCR, if there is no breaking change, you can do `git pull` int `kubo-deployment` and re-run `./deploy-kubernetes.sh`. \n\n\nThe environment as shown in the following figure should be made.\n\n![image](https://user-images.githubusercontent.com/106908/42409160-2c9ec920-8211-11e8-89cc-34b59249e11e.png)\n\n#### Deploy Addons\n\nAddons such as KubeDNS and Kubenetes Dashboard are deployed with errand. Execute the following command.\n\n```bash\nbosh -d cfcr run-errand apply-addons\n```\n\n#### Run smoke tests\n\n```bash\nbosh -d cfcr run-errand smoke-tests\n```\n\n#### Login to CredHub\n\nCredentials information on Kubernetes clusters is stored in CredHub in BOSH Director VM. \nYou need to access CredHub to get the TLS certificate and admin's password.\n\nCreate a script to log in to CredHub.\n\n```bash\ncat \u003c\u003c'EOF' \u003e credhub-login.sh\n#!/bin/bash\ncredhub login \\\n        -s ${BOSH_ENVIRONMENT}:8844 \\\n        --client-name=credhub-admin \\\n        --client-secret=$(bosh int ./bosh-aws-creds.yml --path /credhub_admin_client_secret) \\\n        --ca-cert \u003c(bosh int ./bosh-aws-creds.yml --path /uaa_ssl/ca) \\\n        --ca-cert \u003c(bosh int ./bosh-aws-creds.yml --path /credhub_ca/ca)\nEOF\nchmod +x credhub-login.sh\n```\n\nRun the script and log in to CredHub.\n\n```bash\n./credhub-login.sh\n```\n\nSince the access token to CredHub expires in one hour, login again when it expires.\n\n#### Access to Kubernetes\n\nAcquire admin's password from CredHub.\n\n```bash\nadmin_password=$(credhub get -n /bosh-aws/cfcr/kubo-admin-password | bosh int - --path=/value)\n```\n\nObtain the TLS CA certificate of the Master API.\n\n```bash\ntmp_ca_file=\"$(mktemp)\"\ncredhub get -n /bosh-aws/cfcr/tls-kubernetes | bosh int - --path=/value/ca \u003e \"${tmp_ca_file}\"\n```\n\nSet context for `kubectl`.\n\n```bash\ncluster_name=\"cfcr-aws\"\nuser_name=\"admin-aws\"\ncontext_name=\"cfcr-aws\"\n\nkubectl config set-cluster \"${cluster_name}\" \\\n  --server=\"https://${master_lb_ip_address}:8443\" \\\n  --certificate-authority=\"${tmp_ca_file}\" \\\n  --embed-certs=true\n\nkubectl config set-credentials \"${user_name}\" --token=\"${admin_password}\"\n\nkubectl config set-context \"${context_name}\" --cluster=\"${cluster_name}\" --user=\"${user_name}\"\n\nkubectl config use-context \"${context_name}\"\n```\n\nCheck the cluster info by `kubectl cluster-info`.\n\n```\n$ kubectl cluster-info\nKubernetes master is running at https://demo-cfcr-api-658626716.ap-northeast-1.elb.amazonaws.com:8443\nHeapster is running at https://demo-cfcr-api-658626716.ap-northeast-1.elb.amazonaws.com:8443/api/v1/namespaces/kube-system/services/heapster/proxy\nKubeDNS is running at https://demo-cfcr-api-658626716.ap-northeast-1.elb.amazonaws.com:8443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\nmonitoring-influxdb is running at https://demo-cfcr-api-658626716.ap-northeast-1.elb.amazonaws.com:8443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy\n```\n\nSince the Master API is attached to the Internet Facing ELB,  the contents of `~/.kube/config` can be also used in the laptop.\n\nFinally, the EC2 instances used in this article are as follows.\n\n![image](https://user-images.githubusercontent.com/106908/40376301-f97e7fa6-5e28-11e8-99cf-e40fd3a309ff.png)\n\n### Scale out the k8s cluster\n\nLet's scale out worker to 2 and master to 3 (must be odd).\n\n```yaml\ncat \u003c\u003cEOF \u003e ops-files/kubernetes-worker.yml\n- type: replace\n  path: /instance_groups/name=worker/instances\n  value: 2\nEOF\n```\n\n```bash\ncat \u003c\u003c'EOF' \u003e deploy-kubernetes.sh\n#!/bin/bash\nbosh deploy -d cfcr kubo-deployment/manifests/cfcr.yml \\\n    -o kubo-deployment/manifests/ops-files/addons-spec.yml \\\n    -o kubo-deployment/manifests/ops-files/iaas/aws/lb.yml \\\n    -o kubo-deployment/manifests/ops-files/iaas/aws/cloud-provider.yml \\\n    -o kubo-deployment/manifests/ops-files/use-coredns.yml \\\n    -o ops-files/kubernetes-kubo-0.21.0.yml \\\n    -o ops-files/kubernetes-worker.yml \\\n    -o ops-files/kubernetes-master-lb.yml \\\n    --var-file addons-spec=\u003c(for f in `ls specs/*.yml`;do cat $f;echo;echo \"---\";done) \\\n    -v kubernetes_cluster_tag=${kubernetes_cluster_tag} \\\n    -v kubernetes_master_host=${master_lb_ip_address} \\\n    --no-redact\nEOF\n```\n\n```\n./deploy-kubernetes.sh \n```\n\nRun `bosh vms` to see the VM list.\n\n```\n$ bosh -d cfcr vms\nUsing environment '10.0.8.252' as client 'admin'\n\nTask 18. Done\n\nDeployment 'cfcr'\n\nInstance                                     Process State  AZ  IPs        VM CID               VM Type        Active  \nmaster/0c9bf70c-db82-482d-b38f-fd05dfe0819d  running        z1  10.0.8.4   i-0662d0d4a543d63ba  small          true  \nmaster/517f31c1-d1a5-438d-b508-a0c45f0be822  running        z2  10.0.9.4   i-05acda1e48ce5da70  small          true  \nmaster/d88a8853-d92e-4746-b7a4-c3e36fe741fd  running        z3  10.0.10.4  i-0e22869ccc18676c1  small          true  \nworker/d9808c82-0e18-4a45-87ce-c57b9874db2f  running        z1  10.0.8.5   i-0da422703dd4ecb8c  small-highmem  true  \nworker/ebcd08bf-eafb-403e-b0e4-c849971f4754  running        z2  10.0.9.5   i-05bb0c65aca895cb9  small-highmem  true  \n\n5 vms\n\nSucceeded\n```\n\nThe environment as shown in the following figure should be made.\n![image](https://user-images.githubusercontent.com/106908/42409164-37fa45ba-8211-11e8-851d-051914841641.png)\n\nEC2 console will look like following:\n\n![image](https://user-images.githubusercontent.com/106908/42382726-2926b48e-8170-11e8-9131-5c70d718caba.png)\n\n### Enable UAA integration\n\n[Enable UAA](enable-uaa.md)\n\n### Destroy CFCR\n\nDelete the used environment.\n\n#### Destroy the Kubernetes\n\nDelete the Kubernetes cluster with the following command. \nNote that ELB for Service provisioned by Kubernetes and EBS for Persistent Volume are out of BOSH management, so delete them with `kubectl` command in advance.\n\n```bash\nbosh -d cfcr delete-deployment\nbosh -n clean-up --all\n```\n\n#### Delete the BOSH Director\n\nDelete the BOSH Director with the following command.\n\n```bash\neval \"$(sed 's/create-env/delete-env/' deploy-bosh.sh)\"\n```\n\n#### Delete the AWS environment\n\nDelete the AWS environment with the following command.\n\n```bash\nterraform destroy\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaking%2Fcfcr-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaking%2Fcfcr-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaking%2Fcfcr-aws/lists"}