{"id":16179241,"url":"https://github.com/jonashackt/crossplane-kuttl","last_synced_at":"2026-04-24T16:03:08.880Z","repository":{"id":232226784,"uuid":"783764614","full_name":"jonashackt/crossplane-kuttl","owner":"jonashackt","description":"Example project showing how to use KUTTL to create e2e integration tests with Crossplane","archived":false,"fork":false,"pushed_at":"2026-04-24T10:54:45.000Z","size":262,"stargazers_count":6,"open_issues_count":4,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-04-24T12:34:19.632Z","etag":null,"topics":["aws","crossplane","integration-testing","kubernetes","kuttl"],"latest_commit_sha":null,"homepage":"https://www.codecentric.de/wissens-hub/blog/testing-crossplane-compositions-kuttl","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonashackt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-04-08T14:23:46.000Z","updated_at":"2025-10-10T05:32:34.000Z","dependencies_parsed_at":"2025-08-09T11:19:14.634Z","dependency_job_id":null,"html_url":"https://github.com/jonashackt/crossplane-kuttl","commit_stats":null,"previous_names":["jonashackt/crossplane-objectstorage","jonashackt/crossplane-kuttl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jonashackt/crossplane-kuttl","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fcrossplane-kuttl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fcrossplane-kuttl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fcrossplane-kuttl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fcrossplane-kuttl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonashackt","download_url":"https://codeload.github.com/jonashackt/crossplane-kuttl/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonashackt%2Fcrossplane-kuttl/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":32230421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-24T13:21:15.438Z","status":"ssl_error","status_checked_at":"2026-04-24T13:21:15.005Z","response_time":64,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","crossplane","integration-testing","kubernetes","kuttl"],"created_at":"2024-10-10T05:26:14.765Z","updated_at":"2026-04-24T16:03:08.864Z","avatar_url":"https://github.com/jonashackt.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# crossplane-kuttl\n[![kuttl-crossplane-aws](https://github.com/jonashackt/crossplane-kuttl/actions/workflows/kuttl-crossplane-aws.yml/badge.svg)](https://github.com/jonashackt/crossplane-kuttl/actions/workflows/kuttl-crossplane-aws.yml)\n![crossplane-version](https://img.shields.io/badge/dynamic/yaml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fjonashackt%2Fcrossplane-kuttl%2Fmain%2Fcrossplane%2Finstall%2FChart.yaml\u0026query=%24.dependencies%5B%3A1%5D.version\u0026label=crossplane\u0026color=blue)\n![provider-aws-s3](https://img.shields.io/badge/dynamic/yaml?url=https%3A%2F%2Fraw.githubusercontent.com%2Fjonashackt%2Fcrossplane-kuttl%2Fmain%2Fcrossplane%2Fprovider%2Fupbound-provider-aws-s3.yaml\u0026query=%24.spec.package\u0026label=provider-aws-s3\u0026color=rgb(109%2C%20100%2C%20245))\n[![License](http://img.shields.io/:license-mit-blue.svg)](https://github.com/jonashackt/crossplane-kuttl/blob/master/LICENSE)\n[![renovateenabled](https://img.shields.io/badge/renovate-enabled-yellow)](https://renovatebot.com)\n\nExample project showing how to use KUTTL to create e2e integration tests with Crossplane \n\nWhy I didn't choose to use [uptest](https://github.com/crossplane/uptest):\n\n\u003e __WARNING:__ uptest is a work in progress and hardly ever used by any other than Upbound staff themselves. See [this issue comment](https://github.com/upbound/official-providers-ci/issues/153#issuecomment-1756317685): \"I think we have to be honest and document somewhere that currently uptest is not really usable without surrounding make targets and the build module :)\"\n\nTherefore - and since uptest is based on kuttl - in this repository I went with native kuttl instead for the meantime. \n\n__TLDR;__ run:\n\n```shell\n# Create aws-creds.conf for Crossplane to access AWS\necho \"[default]\naws_access_key_id = $(aws configure get aws_access_key_id)\naws_secret_access_key = $(aws configure get aws_secret_access_key)\n\" \u003e aws-creds.conf\n\n# Run kuttl tests\nkubectl kuttl test\n```\n\nTo run multiple tests, you don't need to setup kind and Crossplane incl. it's Providers every time simply run:\n\n```shell\n# Only once:\nkubectl kuttl test --skip-cluster-delete\n# and the following runs:\nkubectl kuttl test --start-kind=false\n```\n\n\n# The KUbernetes Test TooL (kuttl) \n\nhttps://kuttl.dev/docs/kuttl-test-harness.html\n\n\u003e \"KUTTL is a declarative integration testing harness for testing operators, KUDO, Helm charts, and any other Kubernetes applications or controllers. Test cases are written as plain Kubernetes resources and can be run against a mocked control plane, locally in kind, or any other Kubernetes cluster.\"\n\nSo kuttl reminds me of Molecule for Ansible: A test harness for any Kubernetes application. It sounds like a great fit for Crossplane!\n\n\n### Prerequisites\n\nBe sure to have `kubectl`, `helm` \u0026 `kind` installed.\n\n\n### Install kind\n\nGetting started with kuttl, we want to either use a pre-existing environment. Or start from scratch using kind:\n\n\u003e If you already have a cluster there are no prerequisites. If you want to use the mocked control plane or Kind, you will need Kind (opens new window).\n\n```shell\n# fire up kind\nkind create cluster --image kindest/node:v1.29.2 --wait 5m\n```\n\n\u003e When running with no defined test environment, the default is a preconfigured cluster defined in `$KUBECONFIG`.\n\n\n### Install kuttl kubectl plugin\n\nhttps://kuttl.dev/docs/cli.html \n\n```shell\n# On a Mac\nbrew tap kudobuilder/tap\nbrew install kuttl-cli\n```\n\nAlternatively: One of the best ways to install a kubectl plugin is to use the package manager [krew](https://github.com/kubernetes-sigs/krew). So first install krew via your preferred package manager (see https://krew.sigs.k8s.io/docs/user-guide/setup/install/):\n\n```shell\n# Manjaro Linux\npamac install krew\n```\n\u003e If that gives an error like `go: module cache not found: neither GOMODCACHE nor GOPATH is set`, try without sudo.\n\nNow install kuttl via krew:\n\n```shell\n$ kubectl krew install kuttl\n\nWARNING: To be able to run kubectl plugins, you need to add\nthe following to your ~/.zshrc:\n\n    export PATH=\"${KREW_ROOT:-$HOME/.krew}/bin:$PATH\"\n\nand restart your shell.\n```\n\nAdd the `export ...` statement to your shell configuration as mentioned in the install statement.\n\nNow testdrive kuttl:\n\n```shell\n$ kubectl kuttl --version\nkubectl-kuttl version 0.15.0\n```\n\n\n\n### kuttl building blocks\n\n[kuttl defines the following building blocks](https://kuttl.dev/docs/kuttl-test-harness.html#writing-your-first-test):\n\n* A \"test suite\" (aka `TestSuite`) is comprised of many test cases that are run in parallel. \n* A \"test case\" is a collection of test steps that are run serially - if any test step fails then the entire test case is considered failed.\n* A \"test step\" (aka `TestStep`) defines a set of Kubernetes manifests to apply and a state to assert on (wait for or expect).\n\n\n\n### Create a kuttl test suite: The kuttl-test.yaml\n\nhttps://kuttl.dev/docs/cli.html#examples\n\nFirst in the root of our project we need to create a [`kuttl-test.yaml`](kuttl-test.yaml) defining our `TestSuite`:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestSuite\ntestDirs:\n  - tests/e2e/\nstartKIND: true\nkindContext: crossplane-test\n```\n\nOur pre-created directory must be defined in `testDirs`.\n\nUsing `startKIND: true` kuttl will start a kind cluster with the name `crossplane-test` defined in `kindContext`.\n\nWe should also add the following lines to our `.gitignore` to prevent us from checking in kind logs or `kubeconfig` files:\n\n```shell\nkubeconfig\nkind-logs-*\n```\n\nWe should also create a folder `tests/e2e` where the `e2e` is the name of our test suite:\n\n```shell\nmkdir -p tests/e2e\n```\n\n\n### Install Crossplane in kuttl TestSuite\n\nTo be able to write tests for Crossplane, we need to have it installed in our cluster first.\n\n[kuttl has a `commands` keyword ready](https://kuttl.dev/docs/testing/reference.html#commands) for us in `TestSuite` and `TestStep` objects. Starting with a binary we can execute anything we'd like.\n\nSince we need Crossplane installed and ready for all our tests, we install it in the TestSuite instead of every TestStep.\n\nInside our [`kuttl-test.yaml`](kuttl-test.yaml) we add `command` statements to install Crossplane into the kind test cluster:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestSuite\ncommands:\n  # Install crossplane via Helm Renovate enabled (see https://stackoverflow.com/a/71765472/4964553)\n  - command: helm dependency update crossplane/install\n  - command: helm upgrade --install crossplane --namespace crossplane-system crossplane/install --create-namespace --wait\ntestDirs:\n  - tests/e2e/\nstartKIND: true\nkindContext: crossplane-test\n```\n\nThe [installation of Crossplane works \"Renovate\" enabled via a local Helm Chart](https://stackoverflow.com/a/71765472/4964553) we defined in `crossplane/install` directory:\n\n```yaml\napiVersion: v2\ntype: application\nname: crossplane\nversion: 0.0.0 # unused\nappVersion: 0.0.0 # unused\ndependencies:\n  - name: crossplane\n    repository: https://charts.crossplane.io/stable\n    version: 1.15.1\n```\n\nOnly be sure to add the following to your `.gitignore`:\n\n```shell\n# Exclude Helm charts lock and packages\n**/**/charts\n**/**/Chart.lock\n```\n\nCheck the installation works via:\n\n```shell\nkubectl kuttl test --skip-cluster-delete\n```\n\nThe `--skip-cluster-delete` flag will preserve our `crossplane-test` kind cluster for later runs.\n\nThus a `docker ps` should show the cluster also:\n\n```shell\ndocker ps\nCONTAINER ID   IMAGE                  COMMAND                  CREATED         STATUS              PORTS                       NAMES\n782fa5bb39a9   kindest/node:v1.25.3   \"/usr/local/bin/entr…\"   2 minutes ago   Up About a minute   127.0.0.1:34407-\u003e6443/tcp   crossplane-test-control-plane\n```\n\nYou can even connect to the kind cluster directly setting the `KUBECONFIG` env variable like this:\n\n```shell\nexport KUBECONFIG=\"/home/jonashackt/dev/crossplane-kuttl/kubeconfig\"\n```\n\nNow you should be able to access the kuttle created kind cluster in this current shell:\n\n```shell\n$ kubectl get nodes\nNAME                            STATUS   ROLES           AGE     VERSION\ncrossplane-test-control-plane   Ready    control-plane   4m25s   v1.25.3\n```\n\nTo get your normal `kubectx` config working again, simply run `unset KUBECONFIG`.\n\nIf you need to delete the cluster later, run:\n\n```shell\nkind delete clusters crossplane-test\n```\n\n\n\n### Install AWS Provider in kuttl TestSuite\n\nThe Upbound AWS Provider Family for S3 needed for our objectstorage Composition is located in [`crossplane/provider/upbound-provider-aws-s3.yaml`](crossplane/provider/upbound-provider-aws-s3.yaml):\n\n```yaml\napiVersion: pkg.crossplane.io/v1\nkind: Provider\nmetadata:\n  name: upbound-provider-aws-s3\nspec:\n  package: xpkg.upbound.io/upbound/provider-aws-s3:v1.2.1\n  packagePullPolicy: IfNotPresent # Only download the package if it isn’t in the cache.\n  revisionActivationPolicy: Automatic # Otherwise our Provider never gets activate \u0026 healthy\n  revisionHistoryLimit: 1\n```\n\nThus inside our [`kuttl-test.yaml`](kuttl-test.yaml) we add another `command` statements to install the Provider into the kind test cluster \u0026 wait for it to become healthy:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestSuite\ncommands:\n  # Install crossplane via Helm Renovate enabled (see https://stackoverflow.com/a/71765472/4964553)\n  - command: helm dependency update crossplane-install\n  - command: helm upgrade --install crossplane --namespace crossplane-system crossplane-install --create-namespace --wait\n\n  # Install the crossplane Upbound AWS S3 Provider Family\n  - command: kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml\n  # Wait until AWS Provider is up and running\n  - command: kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3\ntestDirs:\n  - tests/e2e/\nstartKIND: true\nkindContext: crossplane-test\n```\n\n\n\n### Configure AWS Provider in kuttl for testing Resource rendering only (without AWS access)\n\nIt is not always needed to really create resources on AWS through out our tests. It might be enough to just check if the Managed Resources are rendered correctly.\n\nTo get the Crossplane AWS Provider to render the Managed Resources without real AWS connectivity, we use the trick [described here](https://aaroneaton.com/walkthroughs/crossplane-package-testing-with-kuttl/) and create a `Secret` without actual AWS creds. You find it in the file [`crossplane/provider/non-access-secret.yaml`](crossplane/provider/non-access-secret.yaml):\n\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: aws-creds\n  namespace: crossplane-system\ntype: Opaque\nstringData:\n  key: nocreds\n```\n\n\nNow inside our [`kuttl-test.yaml`](kuttl-test.yaml) we add additional `command` statements to create the Secret and configure the AWS Provider:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestSuite\ncommands:\n  # Install crossplane via Helm Renovate enabled (see https://stackoverflow.com/a/71765472/4964553)\n  - command: helm dependency update crossplane/install\n  - command: helm upgrade --install --force crossplane --namespace crossplane-system crossplane/install --create-namespace --wait\n\n  # Install the crossplane Upbound AWS S3 Provider Family\n  - command: kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml\n  # Wait until AWS Provider is up and running\n  - command: kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3\n\n  # Create AWS Provider secret without AWS access\n  - command: kubectl apply -f crossplane/provider/non-access-secret.yaml\n  # Create ProviderConfig to consume the Secret containing AWS credentials\n  - command: kubectl apply -f crossplane/provider/provider-config-aws.yaml\ntestDirs:\n  - tests/e2e/\nstartKIND: true\nkindContext: crossplane-test\n```\n\n\nThe final bit is to configure the AWS Provider via a `ProviderConfig` which is located in [`crossplane/provider/provider-config-aws.yaml`](crossplane/provider/provider-config-aws.yaml):\n\n```yaml\napiVersion: aws.upbound.io/v1beta1\nkind: ProviderConfig\nmetadata:\n  name: default\nspec:\n  credentials:\n    source: Secret\n    secretRef:\n      namespace: crossplane-system\n      name: aws-creds\n      key: creds\n```\n\n\nNow we should be able to run `kubectl kuttl test` and everything sjould be prepared for testing resource rendering with Crossplane:\n\n```shell\n$ kubectl kuttl test\n=== RUN   kuttl\n    harness.go:462: starting setup\n    harness.go:249: running tests with KIND.\n    harness.go:173: temp folder created /tmp/kuttl1667306899\n    harness.go:155: Starting KIND cluster\n    kind.go:66: Adding Containers to KIND...\n    harness.go:275: Successful connection to cluster at: https://127.0.0.1:43619\n    logger.go:42: 10:54:17 |  | running command: [helm dependency update crossplane-install]\n    logger.go:42: 10:54:17 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:17 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:17 |  | Getting updates for unmanaged Helm repositories...\n    logger.go:42: 10:54:18 |  | ...Successfully got an update from the \"https://charts.crossplane.io/stable\" chart repository\n    logger.go:42: 10:54:18 |  | Saving 1 charts\n    logger.go:42: 10:54:18 |  | Downloading crossplane from repo https://charts.crossplane.io/stable\n    logger.go:42: 10:54:18 |  | Deleting outdated charts\n    logger.go:42: 10:54:18 |  | running command: [helm upgrade --install crossplane --namespace crossplane-system crossplane-install --create-namespace --wait]\n    logger.go:42: 10:54:18 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:18 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:18 |  | Release \"crossplane\" does not exist. Installing it now.\n    logger.go:42: 10:54:41 |  | NAME: crossplane\n    logger.go:42: 10:54:41 |  | LAST DEPLOYED: Tue Apr  9 10:54:18 2024\n    logger.go:42: 10:54:41 |  | NAMESPACE: crossplane-system\n    logger.go:42: 10:54:41 |  | STATUS: deployed\n    logger.go:42: 10:54:41 |  | REVISION: 1\n    logger.go:42: 10:54:41 |  | TEST SUITE: None\n    logger.go:42: 10:54:41 |  | running command: [kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml]\n    logger.go:42: 10:54:41 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 created\n    logger.go:42: 10:54:41 |  | running command: [kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3]\n    logger.go:42: 10:55:50 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 condition met\n    logger.go:42: 10:55:50 |  | running command: [kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf]\n    logger.go:42: 10:55:50 |  | secret/aws-creds created\n    logger.go:42: 10:55:50 |  | running command: [kubectl apply -f crossplane/provider/provider-config-aws.yaml]\n    logger.go:42: 10:55:50 |  | providerconfig.aws.upbound.io/default created\n    harness.go:360: running tests\n    harness.go:73: going to run test suite with timeout of 30 seconds for each step\n    harness.go:372: testsuite: tests/e2e/ has 1 tests\n=== RUN   kuttl/harness\n=== RUN   kuttl/harness/objectstorage\n=== PAUSE kuttl/harness/objectstorage\n=== CONT  kuttl/harness/objectstorage\n    logger.go:42: 10:55:50 | objectstorage | Creating namespace: kuttl-test-just-lemur\n    logger.go:42: 10:55:50 | objectstorage | objectstorage events from ns kuttl-test-just-lemur:\n    logger.go:42: 10:55:50 | objectstorage | Deleting namespace: kuttl-test-just-lemur\n=== CONT  kuttl\n    harness.go:405: run tests finished\n    harness.go:513: cleaning up\n    harness.go:522: collecting cluster logs to kind-logs-1712652956\n    harness.go:570: removing temp folder: \"/tmp/kuttl1667306899\"\n    harness.go:576: tearing down kind cluster\n--- PASS: kuttl (133.61s)\n    --- PASS: kuttl/harness (0.00s)\n        --- PASS: kuttl/harness/objectstorage (5.20s)\nPASS\n```\n\n\n\n\n### Create a kuttl test case\n\nhttps://kuttl.dev/docs/kuttl-test-harness.html#create-a-test-case\n\nA kuttl test case is defined by the next directory level. Let's create one for our `objectstorage` composition:\n\n```shell\nmkdir tests/e2e/objectstorage\n```\n\n\n\n### Create a kuttl test step 00-given-install-xrd-composition.yaml\n\nNow we're where we wanted to be in the first place: Writing our first Crossplane-enabled kuttl `TestStep`.\n\nTherefore create a new file called `tests/e2e/objectstorage/00-given-install-xrd-composition.yaml`: \n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestStep\ncommands:\n  # Keep in mind that the apis dir is way up the folder hierachy relative to this TestStep!\n  # Install the XRD\n  - command: kubectl apply -f ../../../apis/objectstorage/definition.yaml\n  # Install the Composition\n  - command: kubectl apply -f ../../../apis/objectstorage/composition.yaml\n  # Wait for XRD to become \"established\"\n  - command: kubectl wait --for condition=established --timeout=20s xrd/xobjectstorages.crossplane.jonashackt.io\n```\n\nHere we install our Composite Resource Definition (XRD) followed by our Composition under test.\n\nWe also wait for the XRD to become `established` before proceeding to the next step.\n\nIf the kuttl logs show errors like `the path \"apis/objectstorage/definition.yaml\" does not exist`:\n\n```shell\n=== RUN   kuttl/harness\n=== RUN   kuttl/harness/objectstorage\n=== PAUSE kuttl/harness/objectstorage\n=== CONT  kuttl/harness/objectstorage\n    logger.go:42: 11:24:22 | objectstorage | Creating namespace: kuttl-test-hopeful-mustang\n    logger.go:42: 11:24:22 | objectstorage/0-given-install-xrd-composition | starting test step 0-given-install-xrd-composition\n    logger.go:42: 11:24:22 | objectstorage/0-given-install-xrd-composition | running command: [kubectl apply -f apis/objectstorage/definition.yaml]\n    logger.go:42: 11:24:22 | objectstorage/0-given-install-xrd-composition | error: the path \"apis/objectstorage/definition.yaml\" does not exist\n    logger.go:42: 11:24:22 | objectstorage/0-given-install-xrd-composition | command failure, skipping 1 additional commands\n   ...\n--- FAIL: kuttl (127.38s)\n    --- FAIL: kuttl/harness (0.00s)\n        --- FAIL: kuttl/harness/objectstorage (5.41s)\nFAIL\n```\n\ncheck the paths in your `command` statements! I missed this also in the first place, since in the TestSuite at [`kuttl-test.yaml`](kuttl-test.yaml) everything worked relatively from the root dir. __BUT__ remember, we're inside `tests/e2e/objectstorage` now! So we need to go up 3 dirs like `../../../apis/objectstorage/definition.yaml` to fetch the correct file.\n\n\n\n### Create a kuttl test step 01-when-applying-claim.yaml\n\nhttps://kuttl.dev/docs/testing/steps.html#format\n\n\u003e \"In a test case's directory, each file that begins with the same index is considered a part of the same test step. All objects inside of a test step are operated on by the test harness simultaneously, so use separate test steps to order operations.\"\"\n\nAs kuttl executes every `00-*` prefixed test step found in the folder before proceeding to the `01-*` one, we can have the `00-given-install-xrd-composition` working as our preparation step for the other steps to come. Terminology is lent from BDD starting with `given`.\n\nAs we already know Crossplane now it's the time to apply the XR or Claim (XRC).\n\nTherefore I created a `tests/e2e/objectstorage/01-when-applying-claim.yaml`, prefixed with a `when` according to BDD practices:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestStep\ncommands:\n  # Create the XR/Claim\n  - command: kubectl apply -f ../../../examples/objectstorage/claim.yaml\n```\n\nHere we apply our Claim residing in the `examples` dir.\n\n\n\n\n\n### Validate / Assert Resource rendering only (without AWS access)\n\n\u003e It's crucial to use `01-assert` as the name here, to get the assertion beeing started after the Claim has been applied.\n\nThe BDD term `then` can't really be integrated into the final assert step. But luckily it's named `tests/e2e/objectstorage/01-assert.yaml`:\n\n```yaml\napiVersion: s3.aws.upbound.io/v1beta1\nkind: Bucket\nmetadata:\n  name: kuttl-test-bucket\nspec:\n  forProvider:\n    region: eu-central-1\n---\napiVersion: s3.aws.upbound.io/v1beta1\nkind: BucketACL\nmetadata:\n  name: kuttl-test-bucket-acl\nspec:\n  forProvider:\n    acl: public-read\n    bucket: kuttl-test-bucket\n    bucketRef:\n      name: kuttl-test-bucket\n    region: eu-central-1\n```\n\nThis test step will be considered completed once our Managed resources rendered are matching the state that we have defined. If the state is not reached by the time the assert's timeout has expired, then the test step and case will be considered failed. \n\nBe sure to define the exact `metadata` as in your Claim, otherwise kuttl won't find it and will give an error like this:\n\n```shell\n    logger.go:42: 15:54:03 | objectstorage/1-when-applying-claim | test step failed 1-when-applying-claim\n    ...\n    logger.go:42: 15:54:03 | objectstorage/1-when-applying-claim | objectstorage.crossplane.jonashackt.io \"managed-upbound-s3\" deleted\n    case.go:364: failed in step 1-when-applying-claim\n    case.go:366: no resources matched of kind: crossplane.jonashackt.io/v1alpha1, Kind=ObjectStorage\n```\n\nNow run our test suite with\n\n```shell\nkubectl kuttl test --skip-cluster-delete\n```\n\nThe `--skip-cluster-delete` will preserve the kind cluster, if our tests failed and thus speep up our development cycle - otherwise kind and the Crossplane installation/configuration will take place in every test run. Since kuttl will create a local `kubeconfig` file, it will also reuse the kind cluster automatically in subsequent runs of:\n\n```shell\nkubectl kuttl test --start-kind=false\n```\n\nA sole `kubectl kuttl test` will give `KIND is already running, unable to start` errors.\n\n\n\n\n#### Assertion errors \u0026 fixes\n\nIf an error occurs like `key is missing from map`:\n\n```shell\ncase.go:366: resource VPC:/: .spec.forProvider.instanceTenancy: key is missing from map\n```\n\none needs to delete that entry from the `01-assert.yaml`.\n\nEven if something appears like \n\n```shell\nresource Subnet:/: .metadata.labels.zone: value mismatch, expected: eu-central-1a != actual: eu-central-1b\n```\n\nFix the `key is missing from map` first! Then the others might disappear.\n\n\nAlso for better readability, we run the kuttl tests one after another by using the `parallel: 1` configuration in the [`kuttl-test.yaml](kuttl-test.yaml):\n\n```yaml\n...\nparallel: 1 # use parallel: 1 to execute one test after another (e.g. for better readability in CI logs)\n```\n\nIf the kuttl output displayes something like `failed to retrieve aws credentials from aws config: failed to refresh cached credentials, static credentials are empty`:\n\n```shell\n+status:\n+  atProvider: {}\n+  conditions:\n+  - lastTransitionTime: \"2024-04-10T13:27:45Z\"\n+    message: 'connect failed: cannot initialize the Terraform plugin SDK async external\n+      client: cannot get terraform setup: failed to retrieve aws credentials from\n+      aws config: failed to refresh cached credentials, static credentials are empty'\n+    reason: ReconcileError\n+    status: \"False\"\n+    type: Synced\n```\n\nThen you might only be using the AWS Provider in kuttl for testing Resource rendering only (without AWS access) missing real AWS credentials. That is no problem and can be ignored, if you only want to check rendering of Managed Resources.\n\n\n\n\n\n\n\n### Integration Testing: Configure AWS Provider in kuttl for testing actual infrastructure provisioning (with real AWS access)\n\nTo get this config working, we need to create a `Secret` containing our AWS credentials Crossplane can later use to access AWS. Therefore create an `aws-creds.conf` file ([aws CLI should be installed and configured](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)):\n\n```shell\necho \"[default]\naws_access_key_id = $(aws configure get aws_access_key_id)\naws_secret_access_key = $(aws configure get aws_secret_access_key)\n\" \u003e aws-creds.conf\n```\n\n\u003e __ATTENTION__: Don't check `aws-creds.conf` into version control. The best is to add `aws-creds.conf` to your `.gitignore`.\n\nNow inside our [`kuttl-test.yaml`](kuttl-test.yaml) we add another `command` statements to create the Secret and configure the AWS Provider:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestSuite\ntimeout: 300 # We definitely need a higher timeout for the external AWS resources to become available\ncommands:\n  # Install crossplane via Helm Renovate enabled (see https://stackoverflow.com/a/71765472/4964553)\n  - command: helm dependency update crossplane/install\n  - command: helm upgrade --install --force crossplane --namespace crossplane-system crossplane/install --create-namespace --wait\n\n  # Install the crossplane Upbound AWS S3 Provider Family\n  - command: kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml\n  # Wait until AWS Provider is up and running\n  - command: kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3\n\n  # Create AWS Provider secret (pre-deleting it to prevent errors like this while re-using the kuttl kind cluster)\n  - command: kubectl delete secret aws-creds -n crossplane-system --ignore-not-found\n  - command: kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf\n  # Create ProviderConfig to consume the Secret containing AWS credentials\n  - command: kubectl apply -f crossplane/provider/provider-config-aws.yaml\ntestDirs:\n  - tests/e2e/\nstartKIND: true\nkindContext: crossplane-test\n```\n\nBefore we create the Secret we delete it :) Why? Because we want to omit errors like this:\n\n```shell\nerror: failed to create secret secrets \"aws-creds\" already exists\n```\n\n\u003e See https://stackoverflow.com/a/45881324/4964553 - [The best approach using `dry-run=client`](https://stackoverflow.com/a/45881259/4964553) etc sadly doesn't work with kuttl producing a `error: unknown shorthand flag: 'f' in -f` error.\n\n\nAlso I configured a higher timeout for resources to become available [via the `timeout` configuration of our TestSuite](https://kuttl.dev/docs/testing/reference.html#testassert). Otherwise we'll run into errors like this soon:\n\n```shell\ncase.go:364: failed in step 1-when-applying-claim\n    case.go:366: command \"kubectl wait --for condition=Ready --timeout=180s objectstorage.crossplane.jonashackt.io/managed-upbound-s3\" exceeded 30 sec timeout, context deadline exceeded\n```\n\nThis is only needed if we want our test to create external resources on AWS. If not, you can leave out the explicit timeout setting.\n\n\nThe final bit is to configure the AWS Provider via a `ProviderConfig` which is located in [`crossplane/provider/provider-config-aws.yaml`](crossplane/provider/provider-config-aws.yaml):\n\n```yaml\napiVersion: aws.upbound.io/v1beta1\nkind: ProviderConfig\nmetadata:\n  name: default\nspec:\n  credentials:\n    source: Secret\n    secretRef:\n      namespace: crossplane-system\n      name: aws-creds\n      key: creds\n```\n\n\nNow running `kubectl kuttl test` should run like this preparing everything for testing with Crossplane:\n\n```shell\n$ kubectl kuttl test\n=== RUN   kuttl\n    harness.go:462: starting setup\n    harness.go:249: running tests with KIND.\n    harness.go:173: temp folder created /tmp/kuttl1667306899\n    harness.go:155: Starting KIND cluster\n    kind.go:66: Adding Containers to KIND...\n    harness.go:275: Successful connection to cluster at: https://127.0.0.1:43619\n    logger.go:42: 10:54:17 |  | running command: [helm dependency update crossplane-install]\n    logger.go:42: 10:54:17 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:17 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:17 |  | Getting updates for unmanaged Helm repositories...\n    logger.go:42: 10:54:18 |  | ...Successfully got an update from the \"https://charts.crossplane.io/stable\" chart repository\n    logger.go:42: 10:54:18 |  | Saving 1 charts\n    logger.go:42: 10:54:18 |  | Downloading crossplane from repo https://charts.crossplane.io/stable\n    logger.go:42: 10:54:18 |  | Deleting outdated charts\n    logger.go:42: 10:54:18 |  | running command: [helm upgrade --install crossplane --namespace crossplane-system crossplane-install --create-namespace --wait]\n    logger.go:42: 10:54:18 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:18 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 10:54:18 |  | Release \"crossplane\" does not exist. Installing it now.\n    logger.go:42: 10:54:41 |  | NAME: crossplane\n    logger.go:42: 10:54:41 |  | LAST DEPLOYED: Tue Apr  9 10:54:18 2024\n    logger.go:42: 10:54:41 |  | NAMESPACE: crossplane-system\n    logger.go:42: 10:54:41 |  | STATUS: deployed\n    logger.go:42: 10:54:41 |  | REVISION: 1\n    logger.go:42: 10:54:41 |  | TEST SUITE: None\n    logger.go:42: 10:54:41 |  | running command: [kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml]\n    logger.go:42: 10:54:41 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 created\n    logger.go:42: 10:54:41 |  | running command: [kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3]\n    logger.go:42: 10:55:50 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 condition met\n    logger.go:42: 10:55:50 |  | running command: [kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf]\n    logger.go:42: 10:55:50 |  | secret/aws-creds created\n    logger.go:42: 10:55:50 |  | running command: [kubectl apply -f crossplane/provider/provider-config-aws.yaml]\n    logger.go:42: 10:55:50 |  | providerconfig.aws.upbound.io/default created\n    harness.go:360: running tests\n    harness.go:73: going to run test suite with timeout of 30 seconds for each step\n    harness.go:372: testsuite: tests/e2e/ has 1 tests\n=== RUN   kuttl/harness\n=== RUN   kuttl/harness/objectstorage\n=== PAUSE kuttl/harness/objectstorage\n=== CONT  kuttl/harness/objectstorage\n    logger.go:42: 10:55:50 | objectstorage | Creating namespace: kuttl-test-just-lemur\n    logger.go:42: 10:55:50 | objectstorage | objectstorage events from ns kuttl-test-just-lemur:\n    logger.go:42: 10:55:50 | objectstorage | Deleting namespace: kuttl-test-just-lemur\n=== CONT  kuttl\n    harness.go:405: run tests finished\n    harness.go:513: cleaning up\n    harness.go:522: collecting cluster logs to kind-logs-1712652956\n    harness.go:570: removing temp folder: \"/tmp/kuttl1667306899\"\n    harness.go:576: tearing down kind cluster\n--- PASS: kuttl (133.61s)\n    --- PASS: kuttl/harness (0.00s)\n        --- PASS: kuttl/harness/objectstorage (5.20s)\nPASS\n```\n\n\n\n\n### Validate / Assert for testing actual infrastructure provisioning (with real AWS access)\n\n\u003e It's crucial to use `01-assert` as the name here, to get the assertion beeing started after the Claim has been applied.\n\nThe BDD term `then` can't really be integrated into the final assert step. But luckily it's named `tests/e2e/objectstorage/01-assert.yaml`:\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestAssert\ntimeout: 30 # override test suite's long timeout again to have fast results in assertion\n# Clean up AWS resources if something goes wrong, see https://kuttl.dev/docs/testing/reference.html#collectors\ncollectors:\n- type: command\n  command: kubectl delete -f ../../../examples/objectstorage/claim.yaml\n---\napiVersion: s3.aws.upbound.io/v1beta1\nkind: Bucket\nmetadata:\n  name: kuttl-test-bucket\nspec:\n  forProvider:\n    region: eu-central-1\n---\napiVersion: s3.aws.upbound.io/v1beta1\nkind: BucketACL\nmetadata:\n  name: kuttl-test-bucket-acl\nspec:\n  forProvider:\n    acl: public-read\n    bucket: kuttl-test-bucket\n    bucketRef:\n      name: kuttl-test-bucket\n    region: eu-central-1\n```\n\nThis test step will be considered completed once our Managed resources rendered are matching the state that we have defined. If the state is not reached by the time the assert's timeout has expired, then the test step and case will be considered failed. \n\n\u003e __The following is only necessary for real external AWS infrastructure:__\n\nUsing [an explicit `TestAssert` definition here](https://kuttl.dev/docs/testing/reference.html#testassert) we're able to override the timeout again here to enable a faster test cycle. Otherwise the assertion would also wait for 300 seconds as defined in the test suite above.\n\nAlso we use a collector to make sure, a cleanup step is also run [in case of an error](https://kuttl.dev/docs/testing/reference.html#collectors).\n\nBe sure to define the exact `metadata` as in your Claim, otherwise kuttl won't find it and will give an error like this:\n\n```shell\n    logger.go:42: 15:54:03 | objectstorage/1-when-applying-claim | test step failed 1-when-applying-claim\n    ...\n    logger.go:42: 15:54:03 | objectstorage/1-when-applying-claim | objectstorage.crossplane.jonashackt.io \"managed-upbound-s3\" deleted\n    case.go:364: failed in step 1-when-applying-claim\n    case.go:366: no resources matched of kind: crossplane.jonashackt.io/v1alpha1, Kind=ObjectStorage\n```\n\nNow run our test suite with\n\n```shell\nkubectl kuttl test --skip-cluster-delete\n```\n\nThe `--skip-cluster-delete` will preserve the kind cluster, if our tests failed and thus speep up our development cycle - otherwise kind and the Crossplane installation/configuration will take place in every test run. Since kuttl will create a local `kubeconfig` file, it will also reuse the kind cluster automatically in subsequent runs of:\n\n```shell\nkubectl kuttl test --start-kind=false\n```\n\nA sole `kubectl kuttl test` will give `KIND is already running, unable to start` errors.\n\nYou may even watch your AWS console, where the bucket gets created:\n\n![](docs/kuttl-test-bucket-created-aws.png)\n\n\n\n\n\n### Cleanup after assertion\n\n\u003e __The following is only necessary for testing real external AWS infrastructure__\n\nWe should also clean up all resources after the last assertion ran. Therefore let's create a `02-*` step like in [`tests/e2e/objectstorage/02-cleanup.yaml`](tests/e2e/objectstorage/02-cleanup.yaml):\n\n```yaml\napiVersion: kuttl.dev/v1beta1\nkind: TestStep\ncommands:\n  # Cleanup AWS resources\n  - command: kubectl delete -f ../../../examples/objectstorage/claim.yaml\n```\n\nThis should make sure, that our Bucket get's deleted in the end - when everything went fine. If not, we have configured our [collector inside the `TestAssert` config](https://kuttl.dev/docs/testing/reference.html#testassert).\n\n\nA full Crossplane featured kuttl test run should look somehow like this:\n\n```shell\nkubectl kuttl test\n=== RUN   kuttl\n    harness.go:462: starting setup\n    harness.go:249: running tests with KIND.\n    harness.go:173: temp folder created /tmp/kuttl2687417911\n    harness.go:155: Starting KIND cluster\n    kind.go:66: Adding Containers to KIND...\n    harness.go:275: Successful connection to cluster at: https://127.0.0.1:38097\n    logger.go:42: 16:26:23 |  | running command: [helm dependency update crossplane-install]\n    logger.go:42: 16:26:24 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 16:26:24 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 16:26:24 |  | Getting updates for unmanaged Helm repositories...\n    logger.go:42: 16:26:24 |  | ...Successfully got an update from the \"https://charts.crossplane.io/stable\" chart repository\n    logger.go:42: 16:26:24 |  | Saving 1 charts\n    logger.go:42: 16:26:25 |  | Downloading crossplane from repo https://charts.crossplane.io/stable\n    logger.go:42: 16:26:25 |  | Deleting outdated charts\n    logger.go:42: 16:26:25 |  | running command: [helm upgrade --install --force crossplane --namespace crossplane-system crossplane-install --create-namespace --wait]\n    logger.go:42: 16:26:25 |  | WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 16:26:25 |  | WARNING: Kubernetes configuration file is world-readable. This is insecure. Location: /home/jonashackt/dev/crossplane-objectstorage/kubeconfig\n    logger.go:42: 16:26:25 |  | Release \"crossplane\" does not exist. Installing it now.\n    logger.go:42: 16:26:47 |  | NAME: crossplane\n    logger.go:42: 16:26:47 |  | LAST DEPLOYED: Tue Apr  9 16:26:25 2024\n    logger.go:42: 16:26:47 |  | NAMESPACE: crossplane-system\n    logger.go:42: 16:26:47 |  | STATUS: deployed\n    logger.go:42: 16:26:47 |  | REVISION: 1\n    logger.go:42: 16:26:47 |  | TEST SUITE: None\n    logger.go:42: 16:26:47 |  | running command: [kubectl apply -f crossplane/provider/upbound-provider-aws-s3.yaml]\n    logger.go:42: 16:26:47 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 created\n    logger.go:42: 16:26:47 |  | running command: [kubectl wait --for=condition=healthy --timeout=180s provider/upbound-provider-aws-s3]\n    logger.go:42: 16:27:53 |  | provider.pkg.crossplane.io/upbound-provider-aws-s3 condition met\n    logger.go:42: 16:27:53 |  | running command: [kubectl create secret generic aws-creds -n crossplane-system --from-file=creds=./aws-creds.conf]\n    logger.go:42: 16:27:53 |  | secret/aws-creds created\n    logger.go:42: 16:27:53 |  | running command: [kubectl apply -f crossplane/provider/provider-config-aws.yaml]\n    logger.go:42: 16:27:54 |  | providerconfig.aws.upbound.io/default created\n    harness.go:360: running tests\n    harness.go:73: going to run test suite with timeout of 300 seconds for each step\n    harness.go:372: testsuite: tests/e2e/ has 1 tests\n=== RUN   kuttl/harness\n=== RUN   kuttl/harness/objectstorage\n=== PAUSE kuttl/harness/objectstorage\n=== CONT  kuttl/harness/objectstorage\n    logger.go:42: 16:27:54 | objectstorage | Creating namespace: kuttl-test-many-bengal\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | starting test step 0-given-install-xrd-composition\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | running command: [kubectl apply -f ../../../apis/objectstorage/definition.yaml]\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | compositeresourcedefinition.apiextensions.crossplane.io/xobjectstorages.crossplane.jonashackt.io created\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | running command: [kubectl apply -f ../../../apis/objectstorage/composition.yaml]\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | composition.apiextensions.crossplane.io/objectstorage-composition created\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | running command: [kubectl wait --for condition=established --timeout=20s xrd/xobjectstorages.crossplane.jonashackt.io]\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | compositeresourcedefinition.apiextensions.crossplane.io/xobjectstorages.crossplane.jonashackt.io condition met\n    logger.go:42: 16:27:54 | objectstorage/0-given-install-xrd-composition | test step completed 0-given-install-xrd-composition\n    logger.go:42: 16:27:54 | objectstorage/1-when-applying-claim | starting test step 1-when-applying-claim\n    logger.go:42: 16:27:54 | objectstorage/1-when-applying-claim | running command: [kubectl apply -f ../../../examples/objectstorage/claim.yaml]\n    logger.go:42: 16:27:56 | objectstorage/1-when-applying-claim | objectstorage.crossplane.jonashackt.io/managed-upbound-s3 created\n    logger.go:42: 16:27:56 | objectstorage/1-when-applying-claim | running command: [kubectl wait --for condition=Ready --timeout=180s objectstorage.crossplane.jonashackt.io/managed-upbound-s3]\n    logger.go:42: 16:29:58 | objectstorage/1-when-applying-claim | objectstorage.crossplane.jonashackt.io/managed-upbound-s3 condition met\n    logger.go:42: 16:29:58 | objectstorage/1-when-applying-claim | test step completed 1-when-applying-claim\n    logger.go:42: 16:29:58 | objectstorage/2-cleanup | starting test step 2-cleanup\n    logger.go:42: 16:29:58 | objectstorage/2-cleanup | running command: [kubectl delete -f ../../../examples/objectstorage/claim.yaml]\n    logger.go:42: 16:29:58 | objectstorage/2-cleanup | objectstorage.crossplane.jonashackt.io \"managed-upbound-s3\" deleted\n    logger.go:42: 16:29:58 | objectstorage/2-cleanup | test step completed 2-cleanup\n    logger.go:42: 16:29:58 | objectstorage | objectstorage events from ns kuttl-test-many-bengal:\n    logger.go:42: 16:29:58 | objectstorage | Deleting namespace: kuttl-test-many-bengal\n=== CONT  kuttl\n    harness.go:405: run tests finished\n    harness.go:513: cleaning up\n    harness.go:522: collecting cluster logs to kind-logs-1712673003\n    harness.go:570: removing temp folder: \"/tmp/kuttl2687417911\"\n    harness.go:576: tearing down kind cluster\n--- PASS: kuttl (247.84s)\n    --- PASS: kuttl/harness (0.00s)\n        --- PASS: kuttl/harness/objectstorage (129.44s)\nPASS\n```\n\n\n\n\n# Doing it all with GitHub Actions\n\nIn order to receive all the benefits of testing, we should execute the kuttl tests on a regular basis and on any updates occuring (triggered by Renovate for example). Thus I did all the steps above in GitHub Actions again [`.github/workflows/kuttl-crossplane-aws.yml](.github/workflows/kuttl-crossplane-aws.yml):\n\n```shell\nname: kuttl-crossplane-aws\n\non: [push]\n\n# Secrets configuration is only needed for real external AWS infrastructure\nenv:\n  # AWS\n  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}\n  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}\n  AWS_DEFAULT_REGION: 'eu-central-1'\n\njobs:\n  run-kuttl:\n    runs-on: ubuntu-latest\n    steps:\n      - name: Checkout\n        uses: actions/checkout@master\n\n      # Secrets configuration is only needed for real external AWS infrastructure\n      - name: Prepare AWS access via aws-creds.conf\n        run: |\n          echo \"### Create aws-creds.conf file\"\n          echo \"[default]\n          aws_access_key_id = $AWS_ACCESS_KEY_ID\n          aws_secret_access_key = $AWS_SECRET_ACCESS_KEY\n          \" \u003e aws-creds.conf\n\n      - name: Install kuttl \u0026 run Crossplane featured kuttl tests\n        run: |\n          echo \"### Add homebrew to path as described in https://github.com/actions/runner-images/blob/main/images/linux/Ubuntu2004-Readme.md#notes\"\n          eval \"$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)\"\n        \n          echo \"### Install kuttl via brew\"\n          brew tap kudobuilder/tap\n          brew install kuttl-cli\n\n          echo \"### Let's try to use kuttl\"\n          kubectl kuttl --version\n\n          echo \"### Run Crossplane featured kuttl tests\"\n          kubectl kuttl test\n```\n\n\n\n\n\n\n# Uptest\n\n\u003e __Uptest is CURRENTLY NOT SUPPORTED IN THIS REPO!__ Only saving the following docs for later, [when uptest is really ready to be used by people outside of Upbound](https://github.com/upbound/official-providers-ci/issues/153#issuecomment-1756317685)\n\nhttps://github.com/crossplane/uptest\n\n\u003e The end to end integration testing tool for Crossplane providers and configurations.\n\nUptest is based on https://kuttl.dev/ \u0026 generates a kuttl test case based on the provided input. You can even inspect the generated kuttl test case by checking the temporary test directory which is printed in the beginning of uptest e2e output.\n\n\u003e Uptest expects a running control-plane (a.k.a. k8s + crossplane) where required providers are running and/or required configuration were applied.\n\nSo running uptest without a working Crossplane managed cluster setup isn't possible (like [this](https://github.com/jonashackt/crossplane-aws-azure) or [this](https://github.com/jonashackt/crossplane-argocd) one). This is also stated when running the `uptest e2e` command: \n\n\u003e \"Run e2e tests for manifests by applying them to a control plane and waiting until a given condition is met.\"\n\n\n### Install uptest\n\n\u003e \"Uptest comes as a binary which can be installed from the releases section.\"\n\nStrangely this currently means, the binary is available from https://github.com/upbound/official-providers-ci/releases - but not from https://github.com/crossplane/uptest/releases! \n\n```shell\n# For Mac ARM\ncurl -sfLo uptest \"https://github.com/upbound/official-providers-ci/releases/download/v0.11.1/uptest_darwin-arm64\"\n\n# For Linux / WSL\ncurl -sfLo uptest \"https://github.com/upbound/official-providers-ci/releases/download/v0.11.1/uptest_linux-amd64\"\n\n# any OS\nchmod +x uptest\nsudo mv uptest /usr/local/bin\n``` \n\nIf this went well, the `uptest` command should work on your system:\n\n```shell\n$ uptest e2e --help\nusage: uptest e2e [\u003cflags\u003e] [\u003cmanifest-list\u003e]\n\nRun e2e tests for manifests by applying them to a control plane and waiting until a given condition is met.\n\nFlags:\n  --help                         Show context-sensitive help (also try --help-long and --help-man).\n  --data-source=\"\"               File path of data source that will be used for injection some values.\n  --setup-script=\"\"              Script that will be executed before running tests.\n  --teardown-script=\"\"           Script that will be executed after running tests.\n  --default-timeout=1200         Default timeout in seconds for the test. Timeout could be overridden per resource\n                                 using \"uptest.upbound.io/timeout\" annotation.\n  --default-conditions=\"Ready\"   Comma separated list of default conditions to wait for a successful test.\n                                 Conditions could be overridden per resource using \"uptest.upbound.io/conditions\"\n                                 annotation.\n  --skip-delete                  Skip the delete step of the test.\n  --test-directory=\"/tmp/uptest-e2e\"  \n                                 Directory where kuttl test case will be generated and executed.\n  --only-clean-uptest-resources  While deletion step, only clean resources that were created by uptest\n\nArgs:\n  [\u003cmanifest-list\u003e]  List of manifests. Value of this option will be used\n                     to trigger/configure the tests.The possible usage:\n                     'provider-aws/examples/s3/bucket.yaml,provider-gcp/examples/storage/bucket.yaml': The comma\n                     separated resources are used as test inputs. If this option is not set, 'MANIFEST_LIST' env var\n                     is used as default.\n\n``` \n\n\n\n### Use Uptest\n\nAs mentioned above you need to have a working Crossplane setup in place (like [this](https://github.com/jonashackt/crossplane-aws-azure) or [this](https://github.com/jonashackt/crossplane-argocd) one)!\n\n\u003e The repository https://github.com/upbound/official-providers-ci contains GitHub Action bases examples of workflows using `uptest e2e`.\n\n\u003e See for example this workflow: https://github.com/upbound/official-providers-ci/blob/main/.github/workflows/pr-comment-trigger.yml \n\n\u003e It may be used like this: https://github.com/upbound/configuration-gitops-argocd/blob/main/.github/workflows/e2e.yaml\n\n\n\nSince uptest is based on kuttl, we need to install it before - see https://github.com/upbound/official-providers-ci/issues/153. Otherwise we'll get errors like: \n\n```shell\nuptest e2e examples/objectstorage/claim.yaml --test-directory=\"$PWD/temp/uptest-e2e\" --setup-script=\"test/setup.sh\"\nRunning kuttl tests at /home/jonashackt/dev/crossplane-objectstorage/temp/uptest-e2e\nbash: line 1: : command not found\nuptest: error: cannot run e2e tests successfully: cannot execute tests: kuttl failed: exit status 127\n```\n\n\nBefore executing uptest, we need to define the following environment varibable:\n\n```shell\nexport KUTTL='kubectl kuttl'  \n```\n\n\n\nSo here we go, let's test our simple Bucket composition:\n\n```shell\nuptest e2e examples/objectstorage/claim.yaml --test-directory=\"$PWD/temp/uptest-e2e\" --setup-script=\"test/setup.sh\"\n```\n\nTODO: Didn't get it to work though. Seems that https://github.com/upbound/official-providers-ci/issues/153#issuecomment-1756317685 is correct currently :(\n\n\n# Links\n\nhttps://aaroneaton.com/walkthroughs/crossplane-package-testing-with-kuttl/\n\nhttps://github.com/awslabs/crossplane-on-eks/blob/main/tests/kuttl/test-suite.yaml","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonashackt%2Fcrossplane-kuttl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonashackt%2Fcrossplane-kuttl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonashackt%2Fcrossplane-kuttl/lists"}