{"id":13484190,"url":"https://github.com/cloudfoundry-incubator/kubo-release","last_synced_at":"2025-03-27T16:30:35.617Z","repository":{"id":18548606,"uuid":"68842993","full_name":"cloudfoundry-incubator/kubo-release","owner":"cloudfoundry-incubator","description":"Kubernetes BOSH release","archived":true,"fork":false,"pushed_at":"2022-02-03T08:19:23.000Z","size":27136,"stargazers_count":161,"open_issues_count":35,"forks_count":76,"subscribers_count":57,"default_branch":"master","last_synced_at":"2024-10-30T18:42:23.461Z","etag":null,"topics":["bosh","bosh-release","cfcr-cluster","cloud-foundry","cloud-foundry-container-runtime","k8s","kubernetes","kubo-release"],"latest_commit_sha":null,"homepage":"https://www.cloudfoundry.org/container-runtime/","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudfoundry-incubator.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-09-21T17:48:31.000Z","updated_at":"2024-10-02T18:01:04.000Z","dependencies_parsed_at":"2022-08-07T09:00:49.092Z","dependency_job_id":null,"html_url":"https://github.com/cloudfoundry-incubator/kubo-release","commit_stats":null,"previous_names":[],"tags_count":148,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry-incubator%2Fkubo-release","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry-incubator%2Fkubo-release/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry-incubator%2Fkubo-release/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry-incubator%2Fkubo-release/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudfoundry-incubator","download_url":"https://codeload.github.com/cloudfoundry-incubator/kubo-release/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245882181,"owners_count":20687842,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bosh","bosh-release","cfcr-cluster","cloud-foundry","cloud-foundry-container-runtime","k8s","kubernetes","kubo-release"],"created_at":"2024-07-31T17:01:20.402Z","updated_at":"2025-03-27T16:30:32.931Z","avatar_url":"https://github.com/cloudfoundry-incubator.png","language":"Ruby","funding_links":[],"categories":["Misc"],"sub_categories":["Azure App Service"],"readme":"# Cloud Foundry Container Runtime\nA [BOSH](http://bosh.io/) release for [Kubernetes](http://kubernetes.io).  Formerly named **kubo**.\n\n- **Slack**: #cfcr on https://slack.cloudfoundry.org\n- **Pivotal Tracker**: https://www.pivotaltracker.com/n/projects/2093412\n\n# Table of Contents\n\u003c!-- vscode-markdown-toc --\u003e\n* [Prerequisites](#Prerequisites)\n  * [Hardware Requirements](#HardwareRequirements)\n* [Deploying CFCR](#DeployingCFCR)\n  * [Configuring CFCR](#ConfiguringCFCR)\n  * [Using Proxy with CFCR](#ProxyWithCFCR)\n  * [BOSH Lite](#BOSHLite)\n* [Accessing the CFCR Cluster with kubectl](#AccessingtheCFCRClusterwithkubectl)\n* [Backup \u0026 Restore](#BackupRestore)\n* [Monitoring](#Monitoring)\n* [DNS](#DNS)\n* [Deprecations](#Deprecations)\n  * [Deployment scripts and docs](#Deploymentscriptsanddocs)\n  * [Heapster](#Heapster)\n\n\u003c!-- vscode-markdown-toc-config\n\tnumbering=true\n\tautoSave=true\n\t/vscode-markdown-toc-config --\u003e\n\u003c!-- /vscode-markdown-toc --\u003e\n\n##  \u003ca name='Prerequisites'\u003e\u003c/a\u003ePrerequisites\n- A BOSH Director configured with UAA, Credhub, and [BOSH DNS runtime config](https://raw.githubusercontent.com/cloudfoundry/bosh-deployment/master/runtime-configs/dns.yml). We recommend using [BOSH Bootloader](https://github.com/cloudfoundry/bosh-bootloader) for this.\n- [Latest kubo-deployment tarball](https://github.com/cloudfoundry-incubator/kubo-deployment/releases/latest)\n- Accessing the master:\n  - **Single Master:** Set up a DNS name pointing to your master's IP address\n  - **Multiple Masters:** A TCP load balancer for your master nodes.\n    - Use a TCP load balancer configured to connect to the master nodes on port 8443.\n    - Add healthchecks using either a TCP dial or HTTPS by looking for a `200 OK` response from `/healthz`.\n    - if you have used [BOSH Bootloader](https://github.com/cloudfoundry/bosh-bootloader) on GCP then you need to manually create a firewall rule.  Allow access to port TCP 8443 to VMs in your BBL network tagged `cfcr-master` from your load balancer's IP.\n- Cloud Config with\n  - `vm_types` named `minimal`, `small`, and `small-highmem` (See [cf-deployment](https://github.com/cloudfoundry/cf-deployment) for reference)\n  - `network` named `default`\n  - three availability zones `azs` named `z1`,`z2`,`z3`\n\n  Note: the cloud-config properties can be customized by applying ops-files. See `manifests/ops-files` for some examples.\n  \n  If using loadbalancers then apply the `vm_extension` called `cfcr-master-loadbalancer` to the cloud-config to add the instances to your loadbalancers. See [BOSH documentation](https://bosh.io/docs/cloud-config/#vm-extensions) for information on how to configure loadbalancers.\n\n####  \u003ca name='HardwareRequirements'\u003e\u003c/a\u003eHardware Requirements\nKubernetes uses etcd as its datastore. The official infrastructure requirements and example configurations for the etcd cluster can be found [here](https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/hardware.md).\n\n##  \u003ca name='DeployingCFCR'\u003e\u003c/a\u003eDeploying CFCR\n\n1. Upload the [latest Xenial stemcell](https://bosh.io/stemcells/#ubuntu-xenial) to the director.\n\n1. Untar the kubo-deployment tarball and rename it `kubo-deployment`\n\n1. Deploy\n\n    ##### Option 1. Single Master\n\n\t```bash\n\tcd kubo-deployment\n\n\tbosh deploy -d cfcr manifests/cfcr.yml \\\n\t  -o manifests/ops-files/misc/single-master.yml \\\n\t  -o manifests/ops-files/add-hostname-to-master-certificate.yml \\\n\t  -v api-hostname=[DNS-NAME]\n\t```\n\n    ##### Option 2. Three Masters\n\n\t```bash\n\tcd kubo-deployment\n\n\tbosh deploy -d cfcr manifests/cfcr.yml \\\n\t  -o manifests/ops-files/add-vm-extensions-to-master.yml \\\n\t  -o manifests/ops-files/add-hostname-to-master-certificate.yml \\\n\t  -v api-hostname=[LOADBALANCER-ADDRESS]\n\t```\n\n\t*Note: Loadbalancer address should be the external address (hostname or IP) of the loadbalancer you have configured.*\n\n   Check additional configurations, such as setting Kubernetes cloud provider, in [docs](./docs/cloud-provider.md).\n\n1. Add Kubernetes system components\n\n    ```bash\n    bosh -d cfcr run-errand apply-specs\n    ```\n\n1. Run the following to confirm the cluster is operational\n\n    ```bash\n    bosh -d cfcr run-errand smoke-tests\n    ```\n###  \u003ca name='ConfiguringCFCR'\u003e\u003c/a\u003eConfiguring CFCR\nPlease check out our manifest and ops-files in kube-deployment for examples on how to configure kubo-release.\nAdditionally, we have a [doc page](docs/configuring-kubernetes-properties.md) to describe how to configure Kubernetes components for the release.\n\nCFCR can be deployed with Pod Security Policies. Check for more details in [the\ndoc](docs/pod-security-policy-walkthrough.md)\n\n####  \u003ca name='ProxyWithCFCR'\u003e\u003c/a\u003eConfiguring Proxy for CFCR\nCFCR allows you to configure proxy for all components. Check [recommendations\nfor no proxy settings](docs/using-proxy.md) first.\n\n###  \u003ca name='BOSHLite'\u003e\u003c/a\u003eBOSH Lite\nCFCR clusters on BOSH Lite are intended for development. We run the [deploy_cfcr_lite](https://github.com/cloudfoundry-incubator/kubo-deployment/blob/master/bin/deploy_cfcr_lite) script to provision a cluster with the latest stemcell and master of kubo-release.  This requires that the cloned kubo-release repository can be found from `cd ../kubo-release` from within the kubo-deployment directory.\n\n```\ncd kubo-deployment\n./bin/deploy_cfcr_lite\n```\n##  \u003ca name='AccessingtheCFCRClusterwithkubectl'\u003e\u003c/a\u003eAccessing the CFCR Cluster with kubectl\n\n1. Login to the Credhub Server that stores the cluster's credentials:\n\t```\n\tcredhub login\n\t```\n1. Find the director name by running\n\t```\n\tbosh env\n\t```\n1. Configure the `kubeconfig` for your `kubectl` client:\n\t```\n\tcd kubo-deployment\n\n\t./bin/set_kubeconfig \u003cDIRECTOR_NAME\u003e/cfcr https://[DNS-NAME-OR-LOADBALANCER-ADDRESS]:8443\n\t```\n##  \u003ca name='BackupRestore'\u003e\u003c/a\u003eBackup \u0026 Restore\nWe use [BBR](https://github.com/cloudfoundry-incubator/bosh-backup-and-restore) to perform backups and restores of the etcd node within a CFCR cluster, for both single and three master deployments. Our backup currently takes an etcd snapshot without interruptions to the cluster. However, for restore we take both the kube-apiserver and etcd offline to restore the cluster with the specified snapshot. Restore is a destructive operation that will completely overwrite any existing data on the cluster. For a closer look at the bbr scripts, check out:\n- [cfcr-etcd-release](https://github.com/cloudfoundry-incubator/cfcr-etcd-release/tree/master/jobs/bbr-etcd)\n- [kubo-release](https://github.com/cloudfoundry-incubator/kubo-release/tree/master/jobs/bbr-kube-apiserver)\n\nTo run the `bbr` cli against a CFCR cluster, follow the steps under \"BOSH Deployment\" on the BBR [documentation page](https://docs.cloudfoundry.org/bbr/#bosh-deployment).\n\n##  \u003ca name='Monitoring'\u003e\u003c/a\u003eMonitoring\n\nFollow the recommendations in [etcd's documentation](https://github.com/etcd-io/etcd/blob/master/Documentation/metrics.md) for monitoring etcd\nmetrics.\n\n##  \u003ca name='DNS'\u003e\u003c/a\u003eDNS\n\nBy default CFCR runs with CoreDNS in preference of Kube-DNS.\n\nIf you are migrating from an earlier version of CFCR, Kube-DNS can be removed by running: \n\n`kubectl delete deployment -n kube-system kube-dns`\n\nYou may notice that a `kube-dns` service remains, this is also required by the CoreDNS spec.\n\n##  \u003ca name='Deprecations'\u003e\u003c/a\u003eDeprecations\n\n###  \u003ca name='Deploymentscriptsanddocs'\u003e\u003c/a\u003eDeployment scripts and docs\nCFCR had a set of scripts, including `deploy_bosh` and `deploy_k8s`, that were the primary mechanism we supported to deploy BOSH and Kubernetes clusters. We no longer support these and have removed the corresponding documentation from https://docs-cfcr.cfapps.io\n\nThe BOSH oriented method documented in this README.md is the supported method to deploy Kubernetes clusters with CFCR.\n\n###  \u003ca name='Heapster'\u003e\u003c/a\u003eHeapster\nK8s 1.11 release kicked off the deprecation timeline for the Heapster component, see [here](https://github.com/kubernetes/heapster/blob/master/docs/deprecation.md) for more info. As a result, we're in the process of replacing Heapster with [Metrics Server](https://github.com/kubernetes-incubator/metrics-server) in the upcoming releases of kubo-release.\n\nHeapster can be removed by running: \n\n`kubectl delete deployment -n kube-system heapster`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry-incubator%2Fkubo-release","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudfoundry-incubator%2Fkubo-release","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry-incubator%2Fkubo-release/lists"}