{"id":18637245,"url":"https://github.com/openshift/osde2e","last_synced_at":"2025-04-09T13:03:00.194Z","repository":{"id":37359317,"uuid":"180659395","full_name":"openshift/osde2e","owner":"openshift","description":"Repository of e2e test plumbing for OpenShift Dedicated","archived":false,"fork":false,"pushed_at":"2025-03-28T08:57:48.000Z","size":603601,"stargazers_count":42,"open_issues_count":14,"forks_count":123,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-03-28T10:15:18.811Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/openshift.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-04-10T20:31:51.000Z","updated_at":"2025-03-28T07:38:06.000Z","dependencies_parsed_at":"2024-01-10T03:30:57.615Z","dependency_job_id":"e38ec1f0-d3c1-4255-ad08-8dae31119165","html_url":"https://github.com/openshift/osde2e","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fosde2e","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fosde2e/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fosde2e/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/openshift%2Fosde2e/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/openshift","download_url":"https://codeload.github.com/openshift/osde2e/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045230,"owners_count":21038553,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T05:34:29.595Z","updated_at":"2025-04-09T13:03:00.167Z","avatar_url":"https://github.com/openshift.png","language":"Go","readme":"# OSDe2e\n\n[![GoDoc](https://godoc.org/github.com/openshift/osde2e?status.svg)](https://godoc.org/github.com/openshift/osde2e)\n\n## Introduction\n\nA comprehensive test framework used for Service Delivery to test all aspects of\nManaged OpenShift Clusters ([OpenShift Dedicated]). The data generated by\nthe different test coverage is used to inform product releases and decisions.\n\nOSDe2e key features are:\n\n* Portable test framework that can run anywhere to validate end to end test workflows\n  * Run locally from a developers workstation or from a CI application\n* Supports create/delete different cluster deployment types\n  * ROSA, ROSA Hosted Control Plane (e.g. HyperShift), OSD on AWS\n  * OSD on GCP\n  * Azure (Azure Red Hat OpenShift)\n* Performs cluster health checks to ensure cluster is operational prior to\n  running tests\n* Perform cluster upgrades\n* Captures artifacts for later use, such as\n  * Cluster install/uninstall logs\n  * Test logs\n  * Metrics\n  * Metadata\n  * Must gather artifacts\n* Tests OSD operators along with other OpenShift features from a\n  customer/SRE point of view\n* Provides a test harness to validate [Add Ons][OSDE2E Test Harness]\n\nWhen osde2e is invoked, the standard workflow is followed:\n\n* Load configuration\n* Cluster deployment (when not leveraging an existing cluster)\n* Verify the health of the cluster\n* Run tests (pre upgrade)\n* Collect logs, metrics and metadata\n* Upgrade cluster (when defined)\n* Verify the health of the cluster post upgrade\n* Run tests (post upgrade - when upgrade is defined)\n* Collect logs, metrics and metadata\n* Cluster deprovision (when this is toggled on)\n\n## Prerequisites\n\nPrior to running osde2e, make sure you meet the minimal prerequisites defined below:\n\n* Navigate to [OpenShift Cluster Manager (OCM)][OpenShift Offline Token] to obtain\n  an OpenShift offline token.\n  * Save your token into the environment variable `OCM_TOKEN` for later usage\n* Verify (submit a request if required to [ocm resources]) your Red Hat account\n  has adequate quota for deploying clusters based on your preferred deployment type\n* A go workspace running the minimal version defined in the [go.mod](go.mod)\n\n## Run\n\nOSDe2e can be invoked by one of two ways. Refer to each section below to learn how\nto run it.\n\n### From Source\n\nRunning from source requires you to build the osde2e binary. Follow the steps below\nto do this:\n\n```shell\ngit clone https://github.com/openshift/osde2e.git\ncd osde2e\ngo mod tidy\nmake build\n```\n\nOn completion of the `make build` target, the generated binary will reside in the\ndirectory `./out/`. Where you can then invoke osde2e `./out/osde2e --help`.\n\n### From Container Image\n\nRunning from a container image using a container engine (e.g. docker, podman).\nYou can either build the image locally or consume the public image available on\n[quay.io][OSDE2E Quay Image].\n\n```shell\nexport CONTAINER_ENGINE=\u003cdocker|podman\u003e\n\n# Build Image\nmake build-image\n$CONTAINER_ENGINE run quay.io/redhat-services-prod/osde2e-cicada-tenant/osde2e:latest \u003cargs\u003e\n\n# Pull Image\n$CONTAINER_ENGINE pull quay.io/redhat-services-prod/osde2e-cicada-tenant/osde2e:latest\n$CONTAINER_ENGINE run quay.io/redhat-services-prod/osde2e-cicada-tenant/osde2e:latest \u003cargs\u003e\n```\n\n## Config Input\n\nOSDe2e provides multiple ways for you to provide input to tailor what test workflows\nyou wish to validate. It provides four ways for you to provide input\n(order is lowest to highest precedence):\n\n* Use pre-canned composable default [configs]\n* Provide a custom config\n* Environment variables\n* Command line options\n\n*It is highly recommended to leave sensitive settings as environment variables (e.g. `OCM_TOKEN`).\nThis way the chance of these settings defined in a custom config file are not checked into\nsource control.*\n\n### Pre-Canned Default Configs\n\nThe [configs] package provides pre-canned default configs available for you to use.\nThese config files are named based on what action they are performing. Within the config\nfile can contain multiple settings to tailor osde2e.\n\nExample config [stage](configs/stage.yaml):\n\nThis default config is telling osde2e to use the stage OCM environment.\n\n```yaml\nocm:\n  env: stage\n```\n\nYou can provide N+1 pre-canned configs to osde2e. Example below will deploy\na OSD cluster within the OCM stage environment.\n\n```shell\n./out/osde2e test --configs aws,stage\n```\n\n### Custom Config\n\nThe composable configs consist of a number of small YAML files that can all be loaded together.\nRather than using the built in configs, you can also elect to build your own custom YAML file\nand provide that using the `--custom-config` CLI option.\n\n```shell\nosde2e test --custom-config ./osde2e.yaml\n```\n\nThe custom config below is a basic example for deploying a ROSA STS cluster and running\nall of the OSD operators tests that do not have the informing label associated to them.\n\n```yaml\ndryRun: false\nprovider: rosa\ncloudProvider:\n  providerId: aws\n  region: us-east-1\nrosa:\n  env: stage\n  STS: true\ncluster:\n  name: osde2e\ntests:\n  ginkgoLabelFilter: Operators \u0026\u0026 !Informing\n```\n\nYou can use both pre-canned default configs and your own custom configs:\n\n```shell\n./out/osde2e test --configs aws --custom-config ./osde2e.yaml\n```\n\n### Environment Variables\n\nAny config option can be passed in using environment variables.\nPlease refer to the [config package] for exact environment variable names.\n\nBelow is an example to spin up a OSD cluster and test it:\n\n```shell\nOCM_TOKEN=\u003cocm-token\u003e \\\nOSD_ENV=prod \\\nCLUSTER_NAME=my-cluster \\\nCLUSTER_VERSION=4.12.0 \\\nosde2e test\n```\n\nThese also can be combined with pre-canned default configs and custom configs:\n\n```shell\nOCM_TOKEN=\u003cocm-token\u003e \\\nCLUSTER_VERSION=4.12.0 \\\nosde2e test --configs prod,e2e-suite\n```\n\n```shell\nOCM_TOKEN=\u003cocm-token\u003e \\\nCLUSTER_VERSION=4.12.0 \\\nosde2e test --configs prod,e2e-suite\n```\n\nA list of commonly used environment variables are included in [Config variables].\n\n### Command Line Options\n\nSome configuration settings are also exposed as command-line options.\nA full list can be displayed by providing `--help` after the command.\n\nBelow is an example of using options for the `test` command:\n\n```shell\n./out/osde2e test --cluster-id \u003ccluster-id\u003e \\\n  --provider stage \\\n  --skip-health-check \\\n  --focus-tests \"RBAC Operator\"\n```\n\nAnother example below is you can skip cluster health check, must gather\nas follows.\n\n```shell\nPOLLING_TIMEOUT=1 \\\n./out/osde2e test --cluster-id=\u003ccluster-id\u003e \\\n--configs stage \\\n--skip-must-gather \\\n--skip-health-check \\\n--focus-tests=\"rh-api-lb-test\"\n```\n\nA list of commonly used CLI flags are included in [Config variables].\n\n### Examples\n\nTo see more examples of configuring input for osde2e, refer to the\n[prowgen jobs][OSDE2E ProwGen Job Config] in the OpenShift release repository\nowned by the team. These will be always up to date with the latest changes\nosde2e has to offer.\n\n## Cluster Deployments\n\nOSDe2e provides native support for deploying the following cluster types:\n\n* ROSA\n* ROSA Hosted Control Plane (HyperShift)\n* OSD (OpenShift Dedicated)\n\nYou can have osde2e deploy the cluster if a cluster ID is not provided or\nyou can leverage an existing cluster by giving the cluster ID as input at\nruntime.\n\nYou can also provide it a kubeconfig file and osde2e can attempt to target\nthat cluster.\n\n```shell\nexport TEST_KUBECONFIG=\u003ckubeconfig-file\u003e\n./out/osde2e test \u003cargs\u003e\n```\n\n*It may be possible to test against a non Managed OpenShift cluster\n(a traditional OpenShift Container Platform cluster). Though this will\nrequire you to alter the input settings as non managed clusters will not\nhave certain items applied to them like a Managed cluster would (e.g. OSD\noperators, health checks, etc).*\n\n## Tests\n\nOSDe2e currently holds all core and operator specific tests and are maintained by the CICD team.\nTest types range from core OSD verification, OSD operators to scale/conformance.\n\n*Currently in flight: OSD operator tests will no longer reside in osde2e repository and\nlive directly alongside the operator source code in its repository*\n\n### Selecting Tests To Run\n\nOSDe2e supports a couple different ways you can select which tests you would like to run. Below presents\nthe commonly used methods for this:\n\n* Using the label filter. Labels are ginkgos way to tag test cases. The examples below\n   will tell osde2e to run all tests that have the `E2E` label applied.\n\n```shell\n# Command line option\nosde2e test --label-filter E2E\n\n# Passed in using a custom config file\ntests:\n  ginkgoLabelFilter: E2E\n```\n\n* Using focus strings. Focus strings are ginkgos way to select test cases based on string regex.\n\n```shell\n# Command line option\nosde2e test --focus-tests \"OCM Agent Operator\"\n\n# Custom config file\ntests:\n  focus: \"OCM Agent Operator\"\n```\n\n* Using a combination of labels and focus strings to fine tune your test selection.\n   The examples below tell osde2e to run all ocm agent operator tests and avoid running\n   the upgrade test case.\n\n```shell\n# Command line options\nosde2e test --label-filter \"Operators \u0026\u0026 !Upgrade\" --focus-tests \"OCM Agent Operator\"\n\n# Custom config file\ntests:\n  ginkgoLabelFilter: \"Operators \u0026\u0026 !Upgrade\"\n  focus: \"OCM Agent Operator\"\n```\n\n### Writing Tests\n\nRefer to the [Writing Tests] document for guidelines and standards.\n\nThird-party (Addon) tests are built as containers that spin up and report back results to OSDe2e.\nThese containers are built and maintained by external groups looking to get CI signal for\ntheir product within OSD. The definition of a third-party test is maintained within\nthe `managed-tenants` repo and is returned via the Add-Ons API.\n\nFor more information please see the [OSDE2E Test Harness] repository to learn more\nfor writing add on tests.\n\n## Reporting\n\nEach time osde2e runs it captures as much data that it possible can. Data can include\ncluster/pod logs, prometheus metrics, test data generated, hive version and osde2e version\nto identify any possible flakiness in the environment.\n\nEach time tests are executed a JUnit XML file will be generated to capture all the tests\nthat ran and statistics about them (e.g. pass/fail, duration). These XML files will be later\nused by external applications to present metrics and data for others to see into. An example of\nthis is they are used to present data in [TestGrid Dashboards][TestGrid Dashboard].\n\n## CI Jobs\n\nPeriodic jobs are run daily validating Managed OpenShift clusters, using\n`osde2e`. Check out the [CI Jobs] page to learn more.\n\n[Config variables]:/docs/Config.md\n[configs]:/configs/\n[config package]:/pkg/common/config/config.go\n[ocm resources]: https://gitlab.cee.redhat.com/service/ocm-resources/\n[OSDE2E Quay Image]: quay.io/redhat-services-prod/osde2e-cicada-tenant/osde2e\n[OpenShift Dedicated]: https://docs.openshift.com/dedicated/welcome/index.html\n[OSDE2E Test Harness]: https://github.com/openshift/osde2e-example-test-harness\n[OpenShift Offline Token]:https://cloud.redhat.com/openshift/token\n[OSDE2E ProwGen Job Config]: https://github.com/openshift/release/blob/master/ci-operator/config/openshift/osde2e/openshift-osde2e-main.yaml\n[TestGrid Dashboard]: https://testgrid.k8s.io/redhat-openshift-osd\n[Writing Tests]:/docs/Writing-Tests.md\n[CI Jobs]: /docs/CI-Jobs.md\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift%2Fosde2e","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopenshift%2Fosde2e","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopenshift%2Fosde2e/lists"}