https://github.com/stakater/terraform-azure-openshift
OpenShift on Azure with Terraform!
https://github.com/stakater/terraform-azure-openshift
azure ocp okd openshift origin stakater terraform
Last synced: 10 months ago
JSON representation
OpenShift on Azure with Terraform!
- Host: GitHub
- URL: https://github.com/stakater/terraform-azure-openshift
- Owner: stakater
- Created: 2019-11-18T22:20:59.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-12-15T14:51:29.000Z (over 2 years ago)
- Last Synced: 2024-04-15T04:16:42.750Z (about 2 years ago)
- Topics: azure, ocp, okd, openshift, origin, stakater, terraform
- Language: HCL
- Homepage: https://stakater.com/opensource
- Size: 23.4 KB
- Stars: 5
- Watchers: 6
- Forks: 7
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Terraform Azure Openshift
OpenShift on Azure with Terraform!
[OpenShift Reference Architecture](https://blog.openshift.com/openshift-container-platform-reference-architecture-implementation-guides/) implementation on Azure using Terraform.
## Setup
Make sure you have Terraform(v0.11.x) in your PATH.
Log in into Azure using [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/?view=azure-cli-latest):
```bash
az login
```
### Generate Certificate
To generate certificate to be used by the openshift cluster, use the certs module. Configure the [cert.tfvars](certs/cert.tfvars) file as needed.
To generate the certificate using ACME, do:
```bash
cd certs
terraform apply -var-file=cert.tfvars
```
To get the certificate values, do:
```bash
terraform output public_certificate_pem
terraform output public_certificate_key
terraform output public_certificate_intermediate_pem
```
Once the certificate is generated, you can use these certificates in either terraform-ocp.tfvars or terraform-okd.tfvars files according to your needs
### Create Openshift stack
#### Create service principal
Create a service principal which will allow terraform to create resources on your behalf on azure
```bash
az ad sp create-for-rbac -n {PRINCIPAL_NAME} --password {PASSWORD} --role contributor --scopes /subscriptions/{subscription-id}
```
#### Create resources
You can tweak the [OpenShift inventory file](openshift/provision/template-inventory.yaml). Its rendered, copied and executed on the server using [inventory.tf](openshift/inventory.tf).
To configure OKD, modify the variables in `openshift/terraform-okd.tfvars`, leave the empty variables and replace the variables filled with capital letters and apply:
```bash
cd openshift
terraform apply -var-file=terraform-okd.tfvars
```
To configure OCP, modify the variables in `openshift/terraform-ocp.tfvars`, replace the variables in capital letters and apply:
```bash
cd openshift
terraform apply -var-file=terraform-ocp.tfvars
```
When finished, you will get the public IPs for the Bastion host and for both the External Load Balancer and the Router Load Balancer.
In order to SSH into the Bastion host use the key in the `keys` folder:
```bash
ssh -i keys/bastion.key cloud-user@BASTION_IP
```
The ```oc``` command is configured to be used in the Bastion host.
Also, you can access the other servers from bastion host e.g to access `master1` server execute the following on bastion
```bash
ssh -i openshift.key cloud-user@master1.openshift.local
```
#### Scale up openshift stack
To scale up openshift stack, set the `scale_up` variable to true and add the configuration of new nodes under `OSEv3.children.new_nodes.hosts` in the `openshift/provision/template-inventory.yaml` e.g
```yaml
OSEv3:
children:
new_nodes:
hosts:
infra2.openshift.local:
openshift_node_group_name: node-config-infra
```
and simply do terraform apply.
## Troubleshooting
### Re-executing scripts
- If you have changed some config file in the `openshift/provision` folder and need to re apply the config on the stack then chances are it won't get triggered automatically. This is done by design to avoid automatic deployment e.g in case of scale up. In case you have to re apply the config on the server then use the `terraform taint` command. e.g you have the changed the inventory file and want to re-apply the deploy cluster script then you need to first do:
```bash
terraform taint null_resource.main
```
and then do terraform apply
### Scaling up/ Creating stack fails because server could not be reached
Terraform currently has an [issue](https://github.com/hashicorp/terraform/issues/17101) with resources depending on entire modules. The work around for this is to just re-apply the resource that failed (manually taint resources if needed).
### Openshift sdn pods not starting
On certain AMIs, openshift sdn pods may not start causing the nodes to not become ready. This issue is caused when the network interface is not allowed to be managed by network manager. You can confirm this by reading the file `/etc/sysconfig/network-scripts/ifcfg-eth0` and make sure that `NM_CONTROLLED` is set to `yes`. To automate this, you can add the following task to your standard ansible node config.
```bash
- name: Allow network to be controlled by Network Manager
lineinfile:
dest: /etc/sysconfig/network-scripts/ifcfg-eth0
regexp: '^NM_CONTROLLED=no$'
line: 'NM_CONTROLLED=yes'
backrefs: yes
```
For this repo, it is already added to `openshift/provision/node-config-playbook.yaml`
### ansible-playbook: command not found
```bash
null_resource.bastion_config (remote-exec): Connected!
null_resource.bastion_config (remote-exec): Loaded plugins: langpacks, product-id,
null_resource.bastion_config (remote-exec): : search-disabled-repos,
null_resource.bastion_config (remote-exec): : subscription-manager
null_resource.bastion_config (remote-exec): This system is registered with an entitlement server, but is not receiving updates. You can use subscription-manager to assign subscriptions.
null_resource.bastion_config (remote-exec): No package ansible available.
null_resource.bastion_config (remote-exec): Error: Nothing to do
null_resource.bastion_config (remote-exec): /home/cloud-user/bastion-config.sh: line 4: ansible-playbook: command not found
Error: Error applying plan:
1 error(s) occurred:
* null_resource.bastion_config: error executing "/tmp/terraform_1606393123.sh": Process exited with status 127
```
This issue was due to the existing subscriptions that was causing some packages to not be installed that caused the above issue. This was resolved by removing the existing subsriptions.