{"id":21888627,"url":"https://github.com/swisscom/crossplane-composition-tester","last_synced_at":"2025-03-22T02:24:05.792Z","repository":{"id":224942269,"uuid":"762363968","full_name":"swisscom/crossplane-composition-tester","owner":"swisscom","description":"BDD test framework for the Crossplane compositions implemented with functions","archived":false,"fork":false,"pushed_at":"2025-02-08T19:47:36.000Z","size":257,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-04T22:02:05.582Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swisscom.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-23T16:16:04.000Z","updated_at":"2025-02-22T22:28:26.000Z","dependencies_parsed_at":"2024-03-11T01:40:32.780Z","dependency_job_id":"13e5f45c-7a9f-456a-ac3e-5cbf52faae77","html_url":"https://github.com/swisscom/crossplane-composition-tester","commit_stats":null,"previous_names":["swisscom/crossplane-composition-tester"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fcrossplane-composition-tester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fcrossplane-composition-tester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fcrossplane-composition-tester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swisscom%2Fcrossplane-composition-tester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swisscom","download_url":"https://codeload.github.com/swisscom/crossplane-composition-tester/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244895550,"owners_count":20527905,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-28T11:16:17.828Z","updated_at":"2025-03-22T02:24:05.786Z","avatar_url":"https://github.com/swisscom.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Crossplane Composition Tester\nTest Crossplane compositions implemented with composition functions using BDD (Behavior Driven Development). \nIf you're not familiar with BDD, check out [this](https://cucumber.io/docs/bdd/). \n\n[Cucumber](https://cucumber.io/docs/guides/overview/) is a popular BDD framework for testing. It follows the \n[Gherkin](https://cucumber.io/docs/gherkin/) syntax for writing tests (Given, When, Then).\nTo run the tests, we use [behave](https://behave.readthedocs.io/en/stable/) which \nis a python implementation of Cucumber. We also use Python to implement the steps of the tests.\n\n\n## Installation\nYou need to have the [**Crossplane CLI**](https://docs.crossplane.io/latest/cli/) and [**behave**](https://behave.readthedocs.io/en/latest/) installed. You will also need to install\nthe other dependencies in `requirements.txt` for the python steps implementation of the features.\n```bash\n# Install the python dependencies \npip install -r requirements.txt\n\n# Install the latest version of crossplane cli\ncurl -sL \"https://raw.githubusercontent.com/crossplane/crossplane/master/install.sh\" | sh\n```\n\nYou also need to have **Docker running** since the tests are run inside a docker container.\n\n\n## Quick start\nIn order to get started, we set up an [example composition for a service account](test/pkg/service-account-with-functions/composition.yaml) inside the `test` folder.\n\nThe composition will create the following resources:\n- a `Role`\n- a default `Policy`\n- Any custom policies or predefined policies ARNs provided in the claim\n- a `RolePolicyAttachment` for each policy **once the role and the policies are ready** \n- a `ServiceAccount` object **once the role is ready**\n\nNow you can explore the [feature file](test/composition-tests/service-account-with-functions/service-account.feature) we've written to get an idea\nof how to write features and test different use cases.\n\nYou can run the tests with the following command:\n```bash\nbehave test\n```\n\nYou can check out what options are available for the `behave` command by running\n```bash\nbehave --help\n```\nSome useful options include `--tags` or `-t` to run only scenarios with a certain tag. Some tags related to the severity \nof the scenario or the feature (like \"minor\" or \"critical\") are specific to test reporting tools like [allure](https://github.com/allure-framework/allure-python/blob/master/allure-behave/examples/severity.rst).\n\n**Note**: IDEs like Pycharm have a built-in support for behave and other BDD frameworks will allow you to run the tests directly from the IDE.\n\n## Tests runner\n\nYou can also use our custom `tests_runner.sh` script, that runs the tests against a target folder with feature files and generates cucumber and allure reports that you can view in your CI tool.\n```bash\n./tests_runner.sh test\n```\nHere is an example output with the Jenkins allure plugin\n\n![allure report example 1](docs/assets/allure_report_capture_1.PNG)\n\n![allure report example 2](docs/assets/allure_report_capture_2.PNG)\n\n\n## Motivation\nCrossplane compositions files can become complex and in turn very error-prone.\nSo far, there hasn't been a clear or easy way to test your crossplane compositions in a quick and reproducible way\nother than to test them live in a cluster. This is not ideal for a few reasons:\n- It can be a very slow process depending on the composition\n- It's not easy to debug\n- It does not allow you to test with different combination of inputs (i.e. different configurations).\n\nThis tool aims to solve this problem by allowing you to test your compositions locally and quickly, as well as define tests\nin a way that is easy to read and reason about. This tool relies on two main components:\n- [behave](https://behave.readthedocs.io/en/stable/) - A python implementation of Cucumber, that allows you to define your tests in plain english\nin a BDD (Given, When, Then) format. \n- Crossplane [composition functions](https://docs.crossplane.io/latest/concepts/composition-functions/).\n\nThe Crossplane v1.14 release introduced a new major feature, [composition functions](https://docs.crossplane.io/latest/concepts/composition-functions/).\nComposition functions allow you to define your composition programmatically using a language of your choice (python, go, etc). \nIt came accompanied by the very convenient [render command in the crossplane CLI](https://docs.crossplane.io/latest/concepts/composition-functions/#test-a-composition-that-uses-functions).\nGiven a composition that uses composition functions, this command allows you to render it locally without having to spin up a kubernetes cluster and\nsee the output that will be passed to the providers if you were to run it inside a cluster.\n\nCombining both the power of the crossplane `render` command and the BDD approach, we now have a very powerful and expressive way\nto test our compositions. In the end, whether you write your compositions in yaml or as code with the new composition functions feature,\nthis tool can become very valuable in validating your compositions with different use cases.\n\n## Features\nCheck [the features README in the docs](docs/features.md) to get an overview of how to write features and the rationale behind them in the context of testing Crossplane compositions.\n\n## How it works\nCheck [how this tool works](docs/how_it_works.md).\n\n## CI/CD Integration\n`Behave` has very good integration with test-reporting tools like [Allure](https://allurereport.org/). \n\nFor example in Jenkins, you can install the [Allure plugin](https://plugins.jenkins.io/allure-jenkins-plugin/), and\nthen run the `tests_runner.sh` script which will run the tests with `behave` and generate allure-reports that can \nthen be viewed in Jenkins as seen in the [Quickstart](##quick-start).\n\n## Features Reference\n\nThis is an overview of all the features that are currently supported. More can be added later.\n\n### Given (Arrange)\n\n| Step                                                                                                                                                                                                                                                      | Description                                                                                                                                                                                                                                                                                     |\n|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `Given input claim \u003cCLAIM\u003e`                                                                                                                                                                                                                               | Provide the name of the claim file to be used in the test. By default, the claim file should be named `claim.yaml`. Claims should be stored inside the `resources` subfolder inside each feature folder.                                                                                        |\n| `Given input composition \u003cCOMPOSITION FILE\u003e`                                                                                                                                                                                                                   | Provide the name of the composition file. By default, the composition should be named `composition.yaml`. Compositions should be stored inside the `pkg/\u003cRESOURCE\u003e` directory of the project. This step is OPTIONAL.                                                                            |\n| `Given input composition directory \u003cCOMPOSITION DIRECTORY\u003e and file \u003cCOMPOSITION FILE\u003e`                                                                                                                                                                                                                   | Provide the name of the composition directory and file. The composition file is looked up in `pkg/\u003cCOMPOSITION DIRECTORY\u003e/\u003cCOMPOSITION_FILE\u003e`. This step is OPTIONAL.                                                                        |\n| `Given input functions \u003cFUNCTIONS\u003e`                                                                                                                                                                                                                       | Provide the name of the functions file to be used with the tests. Function files should be stored at the root of the test directory containing the feature files directories of the project (e.g. `test/composition-tests/functions.yaml`). By default, the tests will use the `functions.yaml` file to run the tests. **The functions file should contain all the functions needed to run the tests**. However, one can keep multiple versions of the functions file, and in that case use this step to specify which version to use for the tests. This step is OPTIONAL. |\n| \u003cpre\u003e\u003ccode\u003eGiven input claim is changed with parameters \u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param name \\| param value \\| \u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-1 \\| value-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-2 \\| value-2 \\| \u003c/code\u003e\u003c/pre\u003e                                                 | Updates the claim with the parameters provided in the data table.                                                                                                                                                                                                                               |\n| `Given change all observed resources with status \u003cREADY_STATUS\u003e`                                                                                                                                                                                          | Sets the ready status of all resources in the current observed state                                                                                                                                                                                                                            |\n| \u003cpre\u003e\u003ccode\u003eGiven change following observed resources with status \u003cREADY_STATUS\u003e\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-name \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-2 \\|\u003c/code\u003e\u003c/pre\u003e                                                        | Sets the ready status of the given resources in the current observed state                                                                                                                                                                                                                      |\n| `Given change observed resource \u003cRESOURCE_NAME\u003e with status \u003cREADY_STATUS\u003e`                                                                                                                                                                               | Sets the ready status of the given resource in the current observed state                                                                                                                                                                                                                       |\n| \u003cpre\u003e\u003ccode\u003e\u003cbr/\u003eGiven change observed resource \u003cRESOURCE_NAME\u003e with parameters\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param name \\| param value \\| \u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-1 \\| value-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-2 \\| value-2 \\| \u003c/code\u003e\u003c/pre\u003e                           | Update or set the parameters of the given resource in the current observed state                                                                                                                                                                                                                |\n| \u003cpre\u003e\u003ccode\u003e\u003cbr/\u003eGiven change observed resource \u003cRESOURCE_NAME\u003e with status \u003cREADY_STATUS\u003e and parameters\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param name \\| param value \\| \u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-1 \\| value-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-2 \\| value-2 \\| \u003c/code\u003e\u003c/pre\u003e | Sets the ready status of the given resource in the current observed state and update its parameters                                                                                                                                                                                             |\n\n\n### When (Act)\n\nCurrently, the only supported action is `crossplane renders the composition` which will run the crossplane `render` command with the given inputs from your feature file.\n\n| Step                                      | Description                                                |\n|-------------------------------------------|------------------------------------------------------------|\n| `When crossplane renders the composition` | We apply the claim with the current observed state, if any |\n\n### Then (Assert)\n\n| Step                                                                                                                                                                                                                | Description                                                                                                                                                 |\n|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `Then check that \u003cNUMBER\u003e resources are provisioning`                                                                                                                                                               | Check that a number of resources are being provisioned after we apply a claim.                                                                              |\n| \u003cpre\u003e\u003ccode\u003eThen check that \u003cNUMBER\u003e resources are provisioning and they are\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-name \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| resource-2 \\|\u003c/code\u003e\u003c/pre\u003e                      | Check that a number of resources are being provisioned after we apply a claim and check that their names is equal to the ones you provide in the data table |\n| \u003cpre\u003e\u003ccode\u003eThen check that resource \u003cRESOURCE_NAME\u003e has parameters\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param name \\| param value \\| \u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-1 \\| value-1 \\|\u003c/code\u003e\u003cbr\u003e\u003ccode\u003e\\| param-2 \\| value-2 \\| \u003c/code\u003e\u003c/pre\u003e | Check that a provisioned resource has the parameters you provide in the data table.                                                                         |\n| `Then check that no resources are provisioning`                                                                                                                                                                     | Check that no resources are being provisioned                                                                                                               |\n\n## Built With\n- [Crossplane CLI](https://docs.crossplane.io/latest/cli/): Crossplane CLI tool that includes the `render` command, used extensively in this project (under Apache 2.0 License).\n- [behave](https://pypi.org/project/behave/): BDD framework in Python (under BSD license).\n- [python-benedict](https://pypi.org/project/python-benedict/): Python library for manipulating dicts including keypath support (under MIT License).\n- [allure-behave](https://pypi.org/project/allure-behave/): Allure test reporting for *behave* (under Apache 2.0 License).\n\n\n## Contributing\nPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change. Check [CONTRIBUTING.md](CONTRIBUTING.md) for more.\n\n## License\nThis project is licensed under the Apache 2.0 License - see the [LICENSE](LICENSE) file for details\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswisscom%2Fcrossplane-composition-tester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswisscom%2Fcrossplane-composition-tester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswisscom%2Fcrossplane-composition-tester/lists"}