{"id":19308725,"url":"https://github.com/mtulio/openshift-cluster-orchestrator","last_synced_at":"2025-08-13T07:18:31.682Z","repository":{"id":44117648,"uuid":"400494061","full_name":"mtulio/openshift-cluster-orchestrator","owner":"mtulio","description":"[LAB] OpenShift Cluster Orchestrator (oco) /  benchmark scripts","archived":false,"fork":false,"pushed_at":"2022-06-29T21:18:04.000Z","size":55,"stargazers_count":1,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-06T01:30:05.971Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mtulio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-27T11:59:40.000Z","updated_at":"2022-11-11T17:02:42.000Z","dependencies_parsed_at":"2022-09-03T13:00:13.129Z","dependency_job_id":null,"html_url":"https://github.com/mtulio/openshift-cluster-orchestrator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtulio%2Fopenshift-cluster-orchestrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtulio%2Fopenshift-cluster-orchestrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtulio%2Fopenshift-cluster-orchestrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mtulio%2Fopenshift-cluster-orchestrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mtulio","download_url":"https://codeload.github.com/mtulio/openshift-cluster-orchestrator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240409849,"owners_count":19796797,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T00:16:15.825Z","updated_at":"2025-02-24T03:17:51.246Z","avatar_url":"https://github.com/mtulio.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# openshift-cluster-orchestrator\n\nA tool that runs above `openshift-installer` to orchestrate the configuration pre-install applying configuration in the manifests, or patches on the manifests before triggering the `create cluster`.\n\nThe tool also orchestrates initial benchmark jobs to a fresh cluster, for example running the benchmark in specific components like disks, etcd, node CPU, etc, interacting with projects:\n- openshift-installer\n- FIO\n- benchmark-operator\n\nThe cluster configuration will be created dynamically with a pre-set of cluster profiles (customized), once the cluster is provisioned, it will start a set of pre-defined tests and collect the results locally.\n\nThe scripts run on top of `openshift-installer` using IPI method.\n\nUse cases:\n- benchmark specific components from a cloud provider. Eg:\n  * benchmark AWS disks gp2 and gp3, then compare with Azure disks\n  * test different instance types with a custom Installer binary\n- help on daily tasks spinning-up clusters configuration\n\n## Install\n\n### Requirements\n\n- yq == 2.x\n- jq\n- openshift-installer\n- oc\n\n## Usage\n\nCreate the `.env` file with our credentials.\n\n~~~bash\ncp .env-sample .env\n# adjust .env\n~~~\n\nThe variables must be set:\n- `SSH_PUB_KEYS` : ssh public keys to be added to cluster nodes\n- `PULL_SECRET`  : your pull secret\n- `OPENSHIFT_INSTALL_RELEASE_IMAGE_OVERRIDE` : (optional) if desired, override installer image release\n- `DEFAULT_BASEDOMAIN_AWS` : (optional) if set, the default baseDomain will be used for AWS provider (see config.yaml)\n- `DEFAULT_BASEDOMAIN_AZURE` : (optional) if set, the default baseDomain will be used for Azure provider (see config.yaml)\n\n### `cluster` command\n\nThe `cluster` command will abstract `openshift-installer` tasks, creating dynamic configuration based on cluster profiles, rendering a custom configuration based on pre-defined profiles from `config.yaml`.\n\nThe structure below will be created for each cluster and saved on local storage, `.local/cluster/` directory:\n- `${cluster}_rendered-runner-config.yaml`: dynamic variables created to be used to mount the `install-config.yaml`\n- `${cluster}_install-config.yaml`: rendered `install-config.yaml` created by pre-defined profile (backup of original)\n- `${cluster}/`: directory of `--install-dir` argument from `openshift-installer`. This path will hold all installer stuff\n- `${cluster}-${TIMESTAMP}.tar.xz`: backup of data created before destroy/delete clusters\n\n#### `cluster create`\n\n\u003e similar `openshift-install create cluster`\n\nThis command creates the `install-config.yaml` and runs `openshift-install create cluster`.\n\n**Examples:**\n\n- Create a cluster named `c2awsm5x2xgp2` with profile `aws_m5x2xgp2`\n\n~~~bash\noco cluster create \\\n    --cluster-profile aws_m5x2xgp2 \\\n    --cluster-name c2awsm5x2xgp2 --force\n~~~\n\n- Create a cluster in AWS with manual STS mode:\n\n~~~bash\noco cluster create \\\n    --cluster-profile labccosts \\\n    --cluster-name aws_dev_cco_sts\n~~~\n\n\n#### `cluster create --kubeconfig` or `cluster link`\n\nCommand:\n```\noco cluster create \\\n    --cluster-name c2awsm5x2xgp2 --kubeconfig\n```\n\nLink existing cluster configuration (`$KUBECONFIG`) to `.local/cluster/{cluster-name}/auth/kubeconfig`\n\nCreate a pseudo cluster to be used on this project. This command reads the `${KUBECONFIG}` env var and creates a local configuration on local storage, then it will be possible to run tests in existing clusters / skipping provisioning flow.\n\n#### `cluster setup`\n\n\u003e similar `openshift-install create (manifests|ignition-configs)`\n\nIf you want to render the install-config and create manifests, run the `cluster setup`:\n\n~~~\noco cluster setup \\\n    --cluster-profile aws_m5x2xgp2 \\\n    --cluster-name c2awsm5x2xgp2\n~~~\n\nIf `OPT_CLUSTER_IGNITION_ONLY` is set, the ignition-configs will be created:\n\n~~~bash\nOPT_CLUSTER_IGNITION_ONLY=true\noco cluster setup \\\n    --cluster-profile aws_m5x2xgp2 \\\n    --cluster-name c2awsm5x2xgp2\n~~~\n\nThen, check out the changes and run the `cluster install`.\n\n#### `cluster install`\n\n\u003e similar `openshift-install create cluster`\n\nInstall a cluster from an already created install dir:\n\n~~~\noco cluster install \\\n    --cluster-profile aws_m5x2xgp2 \\\n    --cluster-name c2awsm5x2xgp2\n~~~\n\n#### `cluster destroy`\n\n- Remove a cluster:\n\n~~~bash\noco cluster destroy --cluster-name c2awsm5x2xgp2\n~~~\n\n#### `cluster check / ping`\n\n- Check/ping current cluster\n\n~~~bash\noco cluster ping --cluster-name c2awsm5x2xgp2\n~~~\n\n### `run` command\n\n#### run jobs to the cluster\n\nJobs will bind the cluster (previously created/linked) with `benchmark_profile`, which defines what tasks will run on the cluster. It's possible to run N jobs to the same cluster, the `job-name` should be unique on the project.\n\nAll the results will be collected to local storage path: `.local/results`, if grouped (`job-group`) will be `.local/results/byGroup-${group_name}`\n\nExamples:\n\n1) Create a job named `fio_ebs_gp` binding benchmark profile `fio_psync_singleMaster` to run on the cluster [`ocp_aws`]:\n\n~~~bash\noco run \\\n    --cluster-name ocp_aws \\\n    --job-name fio_ebs_gp \\\n    --benchmark-profile fio_psync_singleMaster\n~~~\n\nThe profile `fio_psync_singleMaster` runs [the recommended](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/benchmark_procedures.html) benckmark using FIO on General Pourpose (gp) EBS on AWS, using the engine `psync` with operations `randread`, `randwrite` and `randrw` (random read or/and write).\n\n2) Create many jobs and group the results into the same directory (`.local/results/byGroup-b3`):\n\n~~~bash\noco run \\\n    --cluster-name c1gp2x1 \\\n    --job-name c1 \\\n    --job-group b3 \\\n    --benchmark-profile fio_allTasks_masterNodes\n\noco run \\\n    --cluster-name c3gp3x1 \\\n    --job-name c3 \\\n    --job-group b3 \\\n    --benchmark-profile fio_allTasks_masterNodes\n~~~\n\n3) override the loop count value from a task definition to `10`, which means to run each task 10 times, instead of `1`(default):\n\n~~~bash\noco run \\\n    --cluster-name c3gp3x1 \\\n    --job-name c3 \\\n    --job-group b3 \\\n    --benchmark-profile fio_allTasks_masterNodes\n    --task-loop 10\n~~~\n\n## Customization\n\nYou can edit any `config.yaml` parameter to create a customized cluster installation.\n\nPlease see below some examples to customize the project.\n\n### create a new cluster profile\n\nYou may need to create a custom cluster profile when:\n- change the instance type\n- change volume type\n- run custom patches\n- run a custom installer\n\n1. create a task function in `src/cluster_jobs`. The function should have the name prefix `task_`\n2. create a task profile in `config.yaml: task_profiles.${name}`\n3. bind the new task to a benchmark profile: `benchmark_profiles.${name}.task_profiles[]`\n\n### create a new task\n\n1. create a task function in `src/cluster_jobs`. Function should have name prefix `task_`\n2. create a task profile in `config.yaml: task_profiles.${name}`\n3. bind the new task to a benchmark profile: `benchmark_profiles.${name}.task_profiles[]`\n\n\n## FAQ\n\nQ: How the tasks will run into the cluster?\nA: Depends on the task executor (function that define the task in `src/cluster_jobs` named `task_${task_name}`). The FIO implementation will run each task profile using `oc debug` thoughts, keeping the connection opened until it is finished. It's possible to schedule a job and wait to finish, but not supported currently.\n\nQ: Are the tasks run in parallel?\nA: No, the tasks will run one-by-one from the `task_profiles` defined on `benchmark_profile`, regardless if has been failed.\n\nQ: Are the jobs run in parallel?\nA: Yes, the jobs will run in parallel on each node defined on `target_node_strategy`\n\nQ: Does this project override the installer?\nA: No, it automates the process to handle customizations (patch manifests, using custom builds, etc) on the installer to run benchmark jobs in different clusters\n\n\n\u003c!--\nToDo:\n### create a new benchmark profile (TODO rename to 'job profile')\n### create a new manifest patch\n\u003e TODO\n--!\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtulio%2Fopenshift-cluster-orchestrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmtulio%2Fopenshift-cluster-orchestrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmtulio%2Fopenshift-cluster-orchestrator/lists"}