{"id":19759564,"url":"https://github.com/linuxacademy/content-openshift-on-openstack","last_synced_at":"2025-06-28T01:35:30.024Z","repository":{"id":85209743,"uuid":"147718774","full_name":"linuxacademy/content-openshift-on-openstack","owner":"linuxacademy","description":"Treva's OpenShift on OpenStack Course","archived":false,"fork":false,"pushed_at":"2020-03-16T17:53:43.000Z","size":242,"stargazers_count":5,"open_issues_count":0,"forks_count":24,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-06T02:01:40.531Z","etag":null,"topics":["okd","openshift","openstack","origin","packstack"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/linuxacademy.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-09-06T18:48:44.000Z","updated_at":"2021-06-01T08:09:17.000Z","dependencies_parsed_at":"2023-04-30T11:36:51.436Z","dependency_job_id":null,"html_url":"https://github.com/linuxacademy/content-openshift-on-openstack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxacademy%2Fcontent-openshift-on-openstack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxacademy%2Fcontent-openshift-on-openstack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxacademy%2Fcontent-openshift-on-openstack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/linuxacademy%2Fcontent-openshift-on-openstack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/linuxacademy","download_url":"https://codeload.github.com/linuxacademy/content-openshift-on-openstack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251701822,"owners_count":21629900,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["okd","openshift","openstack","origin","packstack"],"created_at":"2024-11-12T03:29:56.893Z","updated_at":"2025-04-30T12:33:02.885Z","avatar_url":"https://github.com/linuxacademy.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# General\nThe purpose of this project is to provide a simple, yet flexible deployment of OpenShift on OpenStack using a three step process. This guide assumes you are familiar with OpenStack.\n\n# Contribution\nIf you want to provide additional features, please feel free to contribute via pull requests or any other means.\nWe are happy to track and discuss ideas, topics and requests via 'Issues'.\n\n# Releases\nFor each release of OpenShift a release branch will be created. Starting with OpenShift 3.9 we will follow the OpenShift release version so it is easy to tell what release branch goes with OpenShift version.\n\n* release-1.0 OpenShift 3.7 and earlier\n* release-3.9 OpenShift 3.9\n\nIn addition I would like to metion I borrowed a lot of ideas from two other projects.\n* [OpenShift setup for Hetzner from RH SSA team](https://github.com/RedHat-EMEA-SSA-Team/hetzner-ocp)\n* [OpenShift on OpenStack](https://github.com/redhat-openstack/openshift-on-openstack)\n\n# Pre-requisites\n* Working OpenStack deployment. Tested is OpenStack 12 \u0026 13 (Pike \u0026 Queens) using RDO.\n* RHEL 7 image. Tested is RHEL 7.4.\n* An openstack ssh key for accessing instances.\n* A pre-configured provider (public) network with at least three available floating ips.\n* Flavors configured for OpenShift. These are only recommendations.\n  * ocp.master  (2 vCPU, 4GB RAM, 30 GB Root Disk)\n  * ocp.infra   (4 vCPU, 16GB RAM, 30 GB Root Disk)\n  * ocp.node    (2 vCPU, 4GB RAM, 30 GB Root Disk)\n  * ocp.bastion (1 vCPU, 4GB RAM, 30 GB Root Disk)\n* A router that has the provider network configured as a gateway.\n* Properly configured cinder and nova storage.\n  * Make sure you aren't using default loop back and have disabled disk zeroing in cinder/nova for LVM.\n\nIncrease at least volumes and secgroups in default project quota\n```\n# openstack quota set --volumes 1000 --secgroup-rules 100 \u003cProject Name\u003e\n```\n\nMore information on setting up proper OpenStack environment can be found [here](https://keithtenzer.com/2018/02/05/openstack-12-pike-lab-installation-and-configuration-guide-with-hetzner-root-servers/).\n\n# Tested Deployments\n```Single Master - Non HA```\n\nSingle Master deployment is 1 Master, 1 Infra node and X number of App nodes. This configuration is a non-HA setup, ideal for test environments.\n![](images/openshift_on_openstack_non_ha.PNG)\n\n```Multiple Master - HA```\n\nMultiple Master deployment is 3 Master, 2 Infra node and X number of App nodes. This configuration is an HA setup. By default etcd and registry are not using persistent storage. This would need to be configured post-install manually at this time if those should be persisted.\n![](images/openshift_on_openstack_ha.PNG)\n\n# Install\n![](images/one.png)\n\n```[OpenStack Controller]```\n\nClone Git Repository\n```\n# git clone https://github.com/ktenzer/openshift-on-openstack-123.git\n```\n\nCheckout release branch 3.9\n```\n# git checkout release-3.9\n```\n\nChange dir to repository\n```\n# cd openshift-on-openstack-123\n```\n\nConfigure Parameters\n```\n# cp sample-vars.yml vars.yml\n```\n```\n# vi vars.yml\n---\n### OpenStack Setting ###\nopenstack_user: admin\nopenstack_passwd: \u003cpassword\u003e\nopenstack_ip: \u003cip address\u003e\nopenstack_project: \u003cproject\u003e\ndomain_name: ocp3.lab\ndns_forwarders: [213.133.98.98, 213.133.98.99]\nexternal_network: public\nservice_subnet_cidr: 192.168.1.0/24\nrouter_id: \u003crouter id from 'openstack router list'\u003e\nimage: rhel74\nssh_user: cloud-user\nssh_key_path: /root/admin.pem\nssh_key_name: admin\nstack_name: openshift\nopenstack_version: 13\ncontact: admin@ocp3.lab\nheat_template_path: /root/openshift-on-openstack-123/heat/openshift.yaml\n\n### OpenShift Settings ###\nopenshift_version: 3.9\ndocker_version: 1.13.1\nopenshift_ha: true\nregistry_replicas: 2\nopenshift_user: admin\nopenshift_passwd: \u003cpassword\u003e\n\n### Red Hat Subscription ###\nrhn_username: \u003cuser\u003e\nrhn_password: \u003cpassword\u003e\nrhn_pool: \u003cpool\u003e\n\n### OpenStack Instance Count ###\nmaster_count: 3\ninfra_count: 2\nnode_count: 2\n\n### OpenStack Instance Group Policies ###\n### Set to 'affinity' if only one compute node ###\nmaster_server_group_policies: \"['anti-affinity']\"\ninfra_server_group_policies: \"['anti-affinity']\"\nnode_server_group_policies: \"['anti-affinity']\"\n\n### OpenStack Instance Flavors ###\nbastion_flavor: ocp.bastion\nmaster_flavor: ocp.master\ninfra_flavor: ocp.infra\nnode_flavor: ocp.node\n```\n\nNote: If you want to run a single load balancer (to save floating ips) for masters and infra, instead of default two use following heat template ```heat_template_path: /root/openshift-on-openstack-123/heat/openshift_single_lbaas.yaml```.\n\nAuthenticate OpenStack Credentials\n```\n# source /root/keystonerc_admin\n```\n\nDisable host key checking\n```\n# export ANSIBLE_HOST_KEY_CHECKING=False\n```\n\nDeploy OpenStack Infrastructure for OpenShift\n```\n# ansible-playbook deploy-openstack-infra.yml --private-key=/root/admin.pem -e @vars.yml\n```\n\n![](images/two.png)\n\nGet ip address of the bastion host.\n```\n# openstack stack output show -f value -c output_value openshift ip_address\n\n{\n  \"masters\": [\n    {\n      \"name\": \"master0\",\n      \"address\": \"192.168.1.19\"\n    },\n    {\n      \"name\": \"master1\",\n      \"address\": \"192.168.1.16\"\n    },\n    {\n      \"name\": \"master2\",\n      \"address\": \"192.168.1.15\"\n    }\n  ],\n  \"lb_master\": {\n    \"name\": \"lb_master\",\n    \"address\": \"144.76.134.230\"\n  },\n  \"infras\": [\n    {\n      \"name\": \"infra0\",\n      \"address\": \"192.168.1.10\"\n    },\n    {\n      \"name\": \"infra1\",\n      \"address\": \"192.168.1.11\"\n    }\n  ],\n  \"lb_infra\": {\n    \"name\": \"lb_infra\",\n    \"address\": \"144.76.134.229\"\n  },\n  \"bastion\": {\n    \"name\": \"bastion\",\n    \"address\": \"144.76.134.228\"\n  },\n  \"nodes\": [\n    {\n      \"name\": \"node0\",\n      \"address\": \"192.168.1.6\"\n    },\n    {\n      \"name\": \"node1\",\n      \"address\": \"192.168.1.13\"\n    }\n  ]\n}\n```\n\nSSH to the bastion host using cloud-user and key.\n```\nssh -i /root/admin.pem cloud-user@144.76.134.229\n```\n\n```[Bastion Host]```\n\nChange dir to repository\n```\n# cd openshift-on-openstack-123\n```\n\nDisable host key checking\n```\n[cloud-user@bastion ~]$ export ANSIBLE_HOST_KEY_CHECKING=False\n```\n\nPrepare the nodes for deployment of OpenShift.\n```\n[cloud-user@bastion ~]$ ansible-playbook prepare-openshift.yml --private-key=/home/cloud-user/admin.pem -e @vars.yml\n\nPLAY RECAP *****************************************************************************************\nbastion                    : ok=15   changed=7    unreachable=0    failed=0\ninfra0                     : ok=18   changed=13   unreachable=0    failed=0\ninfra1                     : ok=18   changed=13   unreachable=0    failed=0\nlocalhost                  : ok=7    changed=6    unreachable=0    failed=0\nmaster0                    : ok=18   changed=13   unreachable=0    failed=0\nmaster1                    : ok=18   changed=13   unreachable=0    failed=0\nmaster2                    : ok=18   changed=13   unreachable=0    failed=0\nnode0                      : ok=18   changed=13   unreachable=0    failed=0\nnode1                      : ok=18   changed=13   unreachable=0    failed=0\n```\n\n![](images/three.png)\n\n```[Bastion Host]```\n\nDeploy OpenShift (3.7 or lower).\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/byo/config.yml\nPLAY RECAP *****************************************************************************************\ninfra0.ocp3.lab            : ok=183  changed=59   unreachable=0    failed=0\ninfra1.ocp3.lab            : ok=183  changed=59   unreachable=0    failed=0\nlocalhost                  : ok=12   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=635  changed=265  unreachable=0    failed=0\nmaster1.ocp3.lab           : ok=635  changed=265  unreachable=0    failed=0\nmaster2.ocp3.lab           : ok=635  changed=265  unreachable=0    failed=0\nnode0.ocp3.lab             : ok=183  changed=59   unreachable=0    failed=0\nnode1.ocp3.lab             : ok=183  changed=59   unreachable=0    failed=0\n\n\nINSTALLER STATUS ***********************************************************************************\nInitialization             : Complete\nHealth Check               : Complete\netcd Install               : Complete\nMaster Install             : Complete\nMaster Additional Install  : Complete\nNode Install               : Complete\nHosted Install             : Complete\nService Catalog Install    : Complete\n```\n\nDeploy OpenShift (3.9 or higher)\n\nIn OpenShift 3.9 the playbook names changed and in addition the pre-requisite playbook was added.\n\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml\nPLAY RECAP *****************************************************************************************\ninfra0.ocp3.lab            : ok=61   changed=15   unreachable=0    failed=0\nlocalhost                  : ok=11   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=73   changed=15   unreachable=0    failed=0\nnode0.ocp3.lab             : ok=61   changed=15   unreachable=0    failed=0\n\n\nINSTALLER STATUS ***********************************************************************************\nInitialization             : Complete (0:04:16)\n```\n\nReboot all nodes (masters, infras, nodes and bastion), see Issue 4 below.\n```\n[cloud-user@bastion ~]$ systemctl reboot\n```\n\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml\n\nPLAY RECAP *****************************************************************************************\ninfra0.ocp3.lab            : ok=136  changed=54   unreachable=0    failed=0\nlocalhost                  : ok=12   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=626  changed=259  unreachable=0    failed=0\nnode0.ocp3.lab             : ok=77   changed=27   unreachable=0    failed=0\n\n\nINSTALLER STATUS ***********************************************************************************\nInitialization             : Complete (0:01:40)\nHealth Check               : Complete (0:13:48)\netcd Install               : Complete (0:12:44)\nMaster Install             : Complete (0:23:33)\nMaster Additional Install  : Complete (0:27:00)\nNode Install               : Complete (0:39:11)\nHosted Install             : Complete (0:03:01)\nWeb Console Install        : Complete (0:02:00)\nService Catalog Install    : Complete (0:04:39)\n```\n\nRun post install playbook\n```\n[cloud-user@bastion ~]$ ansible-playbook post-openshift.yml --private-key=/home/cloud-user/admin.pem -e @vars.yml\n\nPLAY RECAP **************************************************************************************************************************\ninfra0                     : ok=4    changed=2    unreachable=0    failed=0\ninfra1                     : ok=4    changed=2    unreachable=0    failed=0\nlocalhost                  : ok=7    changed=6    unreachable=0    failed=0\nmaster0                    : ok=6    changed=4    unreachable=0    failed=0\nmaster1                    : ok=6    changed=4    unreachable=0    failed=0\nmaster2                    : ok=6    changed=4    unreachable=0    failed=0\nnode0                      : ok=4    changed=2    unreachable=0    failed=0\nnode1                      : ok=4    changed=2    unreachable=0    failed=0\n```\n\nLogin in to UI.\n```\nhttps://openshift.144.76.134.226.xip.io:8443\n```\n\n# Optional\nConfigure admin user\n```\n[cloud-user@bastion ~]$ ssh -i /home/cloud-user/admin.pem cloud-user@master0\n```\n\nAuthenticate as system:admin user.\n```\n[cloud-user@master0 ~]$ oc login -u system:admin -n default\n```\n\nMake user OpenShift Cluster Administrator\n```\n[cloud-user@master0 ~]$ oc adm policy add-cluster-role-to-user cluster-admin admin\n```\n\nInstall Metrics\nSet metrics to true in inventory\n```\n[cloud-user@bastion ~]$ vi openshift_inventory\n...\nopenshift_hosted_metrics_deploy=true\n...\n```\n\nRun playbook for metrics for OpenShift 3.7\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/openshift-metrics.yml\nPLAY RECAP **************************************************************************************************************************\ninfra0.ocp3.lab            : ok=45   changed=4    unreachable=0    failed=0\ninfra1.ocp3.lab            : ok=45   changed=4    unreachable=0    failed=0\nlocalhost                  : ok=11   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=48   changed=4    unreachable=0    failed=0\nmaster1.ocp3.lab           : ok=48   changed=4    unreachable=0    failed=0\nmaster2.ocp3.lab           : ok=205  changed=48   unreachable=0    failed=0\nnode0.ocp3.lab             : ok=45   changed=4    unreachable=0    failed=0\nnode1.ocp3.lab             : ok=45   changed=4    unreachable=0    failed=0\n\n\nINSTALLER STATUS ********************************************************************************************************************\nInitialization             : Complete\nMetrics Install            : Complete\n```\n\nRun playbook for metrics for OpenShift 3.9\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/openshift-metrics/config.yml\nPLAY RECAP *****************************************************************************************\ninfra0.ocp3.lab            : ok=0    changed=0    unreachable=0    failed=0\nlocalhost                  : ok=11   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=217  changed=47   unreachable=0    failed=0\nnode0.ocp3.lab             : ok=0    changed=0    unreachable=0    failed=0\n\n\nINSTALLER STATUS ***********************************************************************************\nInitialization             : Complete (0:01:34)\nMetrics Install            : Complete (0:04:37)\n```\n\nInstall Prometheus\nSet prometheus to true in inventory\n```\n[cloud-user@bastion ~]$ vi openshift_inventory\n...\nopenshift_hosted_prometheus_deploy=true\n...\n```\nRun playbook for prometheus for OpenShift 3.9\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/openshift-prometheus/config.yml\nPLAY RECAP *****************************************************************************************\ninfra0.ocp3.lab            : ok=0    changed=0    unreachable=0    failed=0\nlocalhost                  : ok=11   changed=0    unreachable=0    failed=0\nmaster0.ocp3.lab           : ok=217  changed=47   unreachable=0    failed=0\nnode0.ocp3.lab             : ok=0    changed=0    unreachable=0    failed=0\n\nINSTALLER STATUS ***********************************************************************************\nInitialization             : Complete (0:01:34)\nPrometheus Install            : Complete (0:04:37)\n```\n\nInstall Logging\nSet logging to true in inventory\n```\n[cloud-user@bastion ~]$ vi openshift_inventory\n...\nopenshift_hosted_logging_deploy=true\n...\n```\n\nRun playbook for logging OpenShift 3.7\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/byo/openshift-cluster/openshift-logging.yml\n```\n\nRun Playbook for logging OpenShift 3.9\n```\n[cloud-user@bastion ~]$ ansible-playbook -i /home/cloud-user/openshift-inventory --private-key=/home/cloud-user/admin.pem -vv /usr/share/ansible/openshift-ansible/playbooks/openshift-logging/config.yml\n```\n\n# Issues\n## Issue 1: Dynamic storage provisioning using cinder not working\nCurrently using the OpenStack cloud provider requires using Cinder v2 API. Most current OpenStack deployments will default to v3.\n```\nError creating cinder volume: BS API version autodetection failed.\n```\nIf you provision OpenShift volume and it is pending check /var/log/messages on master. If you see this error you need to add following in /etc/origin/cloudprovider/openstack.conf on masters and all nodes then restart node service on node and controller service on master.\n```\n...\n[BlockStorage]\nbs-version=v2\n...\n```\n\nThe post-openshift.yml playbook takes care of setting v2 for cinder automatically.\n\n## Issue 2: Service Catalog Install Fails\n\nThis seems to be general issue with OpenShift 3.7 installer, somtimes API timeout's occur, it can be ignored or you can re-run playbook to install just service catalog.\n\n## Issue 3: Hosted Install Fails\n\nThe registry sometimes fails to complete install due to host resolution of xip.io. Not sure if this is issue in OpenShift 3.7 or environment. Simply re-running hosted playbook resolved the issue and resulted in successful installation.\n\n## Issue 4: Firewalld Error when installing OpenShift 3.9\n\nRHEL 7.5 introduced some changes to firewalld. After installing firewalld, you actually need to reboot your system. The OpenShift 3.9 installer does not account for this, therefore after Step 2, before running OpenShift deployment reboot all nodes and then run OpenShift deployment.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxacademy%2Fcontent-openshift-on-openstack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flinuxacademy%2Fcontent-openshift-on-openstack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flinuxacademy%2Fcontent-openshift-on-openstack/lists"}