{"id":15144492,"url":"https://github.com/helm/acceptance-testing","last_synced_at":"2025-10-23T22:30:38.374Z","repository":{"id":36204200,"uuid":"198440927","full_name":"helm/acceptance-testing","owner":"helm","description":"Acceptance test suite for the Helm client","archived":false,"fork":false,"pushed_at":"2024-05-17T03:20:57.000Z","size":145,"stargazers_count":24,"open_issues_count":14,"forks_count":24,"subscribers_count":17,"default_branch":"main","last_synced_at":"2024-09-27T10:41:39.442Z","etag":null,"topics":["acceptance-testing","chart","charts","cncf","helm","kubernetes","robotframework"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/helm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-23T13:52:56.000Z","updated_at":"2023-11-10T02:50:49.000Z","dependencies_parsed_at":"2023-01-16T23:24:13.773Z","dependency_job_id":null,"html_url":"https://github.com/helm/acceptance-testing","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helm%2Facceptance-testing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helm%2Facceptance-testing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helm%2Facceptance-testing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/helm%2Facceptance-testing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/helm","download_url":"https://codeload.github.com/helm/acceptance-testing/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219867717,"owners_count":16555814,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["acceptance-testing","chart","charts","cncf","helm","kubernetes","robotframework"],"created_at":"2024-09-26T10:41:44.974Z","updated_at":"2025-10-23T22:30:38.035Z","avatar_url":"https://github.com/helm.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Helm Acceptance Tests\n\n[![GitHub Actions status](https://github.com/helm/acceptance-testing/workflows/acceptance-tests/badge.svg)](https://github.com/helm/acceptance-testing/actions)\n\nThis repo contains the source for Helm acceptance tests.\nThe tests are written using [Robot Framework](https://robotframework.org/).\n\n*Note: these tests have only been run against Helm 3 ([dev-v3](https://github.com/helm/helm/tree/dev-v3))*\n\n## Test Summary\n\n### Kubernetes Versions\n\nHelm is tested to work against the following versions of Kubernetes:\n\n\u003c!-- \nTODO\n\nAdd support for 1.16+, getting the following error:\nError: apiVersion \"apps/v1beta1\" in nginx/templates/deployment.yaml is not available\n[1.16.1](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.16.md)\n\nAlso, upgrade to 1.15.4 and 1.14.7\n(see issue on kind: https://github.com/kubernetes-sigs/kind/issues/948)\n\n--\u003e\n\n- [1.15.3](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.15.md)\n- [1.14.6](https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG-1.14.md)\n\nTest suite: [kubernetes_versions.robot](./testsuites/kubernetes_versions.robot)\n\n\n### Shell Completion\n\nHelm's shell completion functionality is tested against the following shells:\n\n- Bash\n- Zsh\n\nTest suite: [shells.robot](./testsuites/shells.robot)\n\n### Helm Repositories\n\nBasic functionality of the chart repository subsystem is tested.\n\nTest suite: [repos.robot](./testsuites/repos.robot)\n\n## System requirements\n\nThe following tools/commands are expected to be present on the base system\nprior to running the tests:\n\n- [kind](https://kind.sigs.k8s.io/)\n- [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/)\n- [python3](https://www.python.org/downloads/)\n- [pip](https://pip.pypa.io/en/stable/installing/)\n- [virtualenv](https://virtualenv.pypa.io/en/latest/installation/)\n\n## Running the tests\n\nFrom the root of this repo, run the following:\n\n```\nmake acceptance\n```\n\nAlternatively, if you have Docker installed, \nthe system requirements above are not needed, and you can run the following\ncommand which will simulate CI:\n```\nmake github-actions-ci-local\n```\n\nNote: by default, the tests will use helm as found on your PATH.\nTo specify a different helm to test, set and export the `ROBOT_HELM_PATH`\nenvironment variable.  For example, if you have helm v2 installed, but want\nto test helm v3 which is located elsewhere; or if you have helm installed\nbut want to test a different development version of helm.\n\n### Selecting which test suites to execute\n\nBy default `make acceptance` will run every test suite (`*.robot` file) present in the directory specified in the environment variable `ROBOT_TEST_ROOT_DIR`.  You can instead specify which test suites to run by setting and exporting variable `ROBOT_RUN_TESTS`.\n\nFor example, to only run the `shells.robot` suite:\n\n```\nROBOT_RUN_TESTS=shells.robot\nmake acceptance\n```\n\nTo specify multiple test suites you can set `ROBOT_RUN_TESTS` to a comma-separated, or space-separated list.  For example:\n\n```\nROBOT_RUN_TESTS=shells.robot,kubernetes_versions.robot\nmake acceptance\n```\n\nYou can use the list format of `ROBOT_RUN_TESTS` as a way to specify the order in which the test suites should be run.  By default (when `ROBOT_RUN_TESTS` is not specified), the test suites are run in alphabetical order.\n\n## Viewing the results\n\nRobot creates an HTML test report describing test successes/failures.\n\nTo view the report, runt the following:\n\n```\nopen .acceptance/report.html\n```\n\nNote: by default, the tests will output to the `.acceptance/` directory.\nTo modify this location, set the `ROBOT_OUTPUT_DIR` environment variable.\n\n## Kubernetes integration\n\nWhen testing Helm against multiple Kubernetes versions,\nnew test clusters are created on the fly (using `kind`),\nwith names in the following format:\n\n```\nhelm-acceptance-test-\u003ctimestamp\u003e-\u003ckube_version\u003e\n```\n\nIf you wish to use an existing `kind` cluster for one\nor more versions, you can set an environment variable for\na given version.\n\nHere is an example of using an existing `kind` cluster\nfor Kubernetes version `1.15.0`:\n\n```\nexport KIND_CLUSTER_1_15_0=\"helm-ac-keepalive-1.15.0\"\n```\n\nA `kind` cluster can be created manually like so:\n\n```\nkind create cluster \\\n  --name=helm-ac-keepalive-1.15.0 \\\n  --image=kindest/node:v1.15.0\n```\n\n## Adding a new test case etc.\n\nAll files ending in `.robot` extension in this directory will be executed.\nAdd a new file describing your test, or, alternatively, add to an existing one.\n\nRobot tests themselves are written in (mostly) plain English, but the Python\nprogramming language can be used in order to add custom keywords etc.\n\nNotice the [lib/](./lib/) directory - this contains Python libraries that\nenable us to work with system tools such as `kind`. The file [common.py](./lib/common.py)\ncontains a base class called `CommandRunner` that you will likely want to\nleverage when adding support for a new external tool.\n\nThe test run is wrapped by [acceptance.sh](./scripts/acceptance.sh) -\nin this file the environment is validated (i.e. check if required tools present). \nIf any additional Python libraries are required for a new library, \nit can be appended to `ROBOT_PY_REQUIRES`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelm%2Facceptance-testing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhelm%2Facceptance-testing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhelm%2Facceptance-testing/lists"}