{"id":19040802,"url":"https://github.com/cloudfoundry/cf-smoke-tests","last_synced_at":"2025-04-06T01:08:10.434Z","repository":{"id":13890219,"uuid":"16588495","full_name":"cloudfoundry/cf-smoke-tests","owner":"cloudfoundry","description":"Smoke tests for CloudFoundry that are safe to run in a production environment","archived":false,"fork":false,"pushed_at":"2025-03-27T00:29:40.000Z","size":32481,"stargazers_count":40,"open_issues_count":0,"forks_count":45,"subscribers_count":109,"default_branch":"main","last_synced_at":"2025-03-30T00:07:07.778Z","etag":null,"topics":["cff-wg-app-runtime-deployments"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudfoundry.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-02-06T18:09:01.000Z","updated_at":"2025-03-27T00:29:43.000Z","dependencies_parsed_at":"2023-01-14T12:30:26.121Z","dependency_job_id":"745f5325-427b-4647-8c08-809a77cc5b22","html_url":"https://github.com/cloudfoundry/cf-smoke-tests","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fcf-smoke-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fcf-smoke-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fcf-smoke-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudfoundry%2Fcf-smoke-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudfoundry","download_url":"https://codeload.github.com/cloudfoundry/cf-smoke-tests/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419860,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cff-wg-app-runtime-deployments"],"created_at":"2024-11-08T22:25:20.970Z","updated_at":"2025-04-06T01:08:10.412Z","avatar_url":"https://github.com/cloudfoundry.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"CF Smoke Tests\n==============\n## Overview\n\n1. [Purpose](#purpose)\n1. [Test Setup](#test-setup)\n    1. [Dependencies](#dependencies)\n    1. [Config](#config)\n1. [Running Tests](#running-tests)\n1. [Contribution To Smoke Tests](#contributing-to-smoke-tests)\n\n\n## Purpose\nSmoke tests are a suite of basic core functionality tests for Cloud Foundry.\nThey are suitable as an initial test against a new or updated deployment to\nreveal fundamental problems with the system.\n\nThere are three tests in this suite, all of which are pretty simple:\n1. `runtime`: Pushes an app and validates that HTTP requests are properly routed to the app.\n2. `logging`: Pushes an app and validates that logs can be fetched for the app.\n3. `isolation_segments`: Entitles an org to an isolation segment and pushes two apps,\n  one to the isolation segment, and one to the shared segment.\n  The test validates that isolated apps are only accessed via the isolated router,\n  and that apps on the shared segment are only accessed via the shared router.\n\nThey are not intended to test more sophisticated functionality of Cloud Foundry\nor to test administrator operations. The [CF Acceptance\nTests](https://github.com/cloudfoundry/cf-acceptance-tests) do perform this\nmore extensive testing, although they are designed to be run as part of a\ndevelopment pipeline and not against production environments.\n\n## Test Setup\n\n### Dependencies\n\nMake sure you have the following installed:\n* [go](https://go.dev/doc/install)\n* [git](http://git-scm.com/)\n* [`cf` CLI](https://github.com/cloudfoundry/cli)\n* [curl](http://curl.haxx.se/)\n\nCheck out a copy of `cf-smoke-tests`. The recommended way to do this is to clone\nthe repo, if you don't have it: `git clone\ngit@github.com:cloudfoundry/cf-smoke-tests.git`. Alternatively, you can simply\n`cd` into a pre-existing `cf-smoke-tests` directory and run `git pull`.\n\nTo run the CF Smoke tests, you will need:\n- a running CF instance\n- an environment variable `$CONFIG` which points to a `.json` file that contains\n  the relevant configuration information.\n\n### Config\n\nBelow is an example `integration_config.json`:\n```json\n{\n  \"suite_name\"                      : \"CF_SMOKE_TESTS\",\n  \"api\"                             : \"api.bosh-lite.com\",\n  \"apps_domain\"                     : \"bosh-lite.com\",\n  \"user\"                            : \"non-admin\",\n  \"password\"                        : \"super-secure\",\n  \"cleanup\"                         : true,\n  \"logging_app\"                     : \"\",\n  \"runtime_app\"                     : \"\",\n  \"enable_windows_tests\"            : false,\n  \"windows_stack\"                   : \"windows\",\n  \"isolation_segment_name\"          : \"is1\",\n  \"isolation_segment_domain\"        : \"is1.bosh-lite.com\",\n  \"enable_isolation_segment_tests\"  : true,\n  \"linux_buildpack_name\"            : \"binary_buildpack\",\n  \"windows_buildpack_name\"          : \"hwc_buildpack\"\n}\n```\nThe following are special case configurations.\n\n#### Credentials\nMust supply one of the following login credentials.\n\n- User credentials\n  ```json\n    \"user\":     \"username\",\n    \"password\": \"password\"\n  ```\n- Client credentials\n  ```json\n    \"client\":        \"client-name\",\n    \"client_secret\": \"client-secret\"\n  ```\n\n#### Skip SSL validation\nIf you are running the tests against bosh-lite or any other environment using\nself-signed certificates, add:\n```json\n  \"skip_ssl_validation\": true\n```\n\n#### Org and space cleanup\nIf you would like to preserve the organization, space, and app created during the\ntests for debugging, add:\n```json\n  \"cleanup\": false\n```\n\n#### Windows\n\nIf you have deployed Windows cells, add:\n```json\n  \"enable_windows_tests\": true,\n  \"windows_stack\":        \"windows\"\n```\nThe valid options for `windows_stack` are `windows2016` and `windows`.\n\n#### Isolation segments\n\nIf you'd like to run isolation segment tests, add:\n```json\n  \"enable_isolation_segment_tests\": true,\n  \"isolation_segment_name\": \"name\",\n  \"isolation_segment_domain\": \"domain\"\n```\nFor more details on how to setup routing for isolation segments, read [this\ndocument](https://docs.cloudfoundry.org/adminguide/routing-is.html).\n\n#### Artifacts directory\nTo store cf cli trace output, set\n```json\n  \"artifacts_directory\": \"/tmp/smoke-artifacts\"\n```\nThe following files may be created:\n```bash\nCF-TRACE-Smoke-1.txt\nCF-TRACE-Smoke-2.txt\njunit-Applications-1.xml\n...\n```\n\n#### Admin vs. Regular User\nSmoke tests can be configured with two types of users.\n\n1. **Regular user**:\nSmoke tests can be configured to run with a non-admin user. If you'd like to use\na non-admin user, it must be able to assign user roles (either `OrgManager` or\n`SpaceManager` roles). Please refer to the [Roles and Permissions for Active\nOrgs](https://docs.cloudfoundry.org/concepts/roles.html#roles-and-permissions-for-active-orgs)\ndocumentation for more information.  In this configuration, organization and\nspace must be created ahead of time and provided as `org` and `space`\nconfiguration properties, respectively.  Also, `use_existing_org` and\n`use_existing_space` must be set to `true`.  This configuration is recommended\nfor tests run against environments run by humans, in particular, production\ndeployments.\n\n2. **Admin user:**\nSmoke tests can be configured to run using admin credentials. Given this\nconfiguration, the tests may or may not use existing resources like orgs and\nspaces, because an admin user can easily create them. This configuration is\nrecommended for tests run against environments created using automation tools,\nfor example, CI (continuous integration) environments on development teams.\n\n**NOTE:**\nIf you are enabling isolation segments tests and would like to use a non-admin\nuser, the existing organization provided as `org` configuration property must be\nentitled to the isolation segment provided by `isolation_segment_name`. In\naddition, you need to create a space inside `org` and provide it as\n`isolation_segment_space`. `isolation_segment_space` must be assigned the\nisolation segment `isolation_segment_name`.\n\n**NOTE:**\nIf the default identity provider for your deployment is not UAA, it is\nrecommended that you set the `origin` configuration property to UAA, and ensure\nthe user credentials that you provide are registered with UAA.\n\n## Running Tests\nTo execute the tests, run:\n```bash\n./bin/test\n```\nInternally the `bin/test` script runs tests using [ginkgo](https://github.com/onsi/ginkgo).\n\nArguments, such as `--keep-going`, `--flake-attempts [int]`, etc., that are\npassed to the test script are passed through to `ginkgo`. For example, to\nexecute tests in parallel one would run:\n```bash\n./bin/test -p\n```\n\n### Seeing command-line output\nTo see verbose output from `cf`, use [ginkgo](https://github.com/onsi/ginkgo)'s `-v` flag:\n```bash\n./bin/test -v\n```\n\n## Contributing to Smoke Tests\n\n### Guidelines\nThe goal of smoke tests is to provide a small, simple set of tests to verify\nbasic deployment configuration. As such, we have some guidelines for\ncontributing new tests to this suite.\n\n#### Creating API resources in the test\nOne basic rule for good test design is not to mock the object under test. We can\ntranslate that idea to a suite like smoke tests in the following way: If smoke\ntests exist to validate deployment configuration, then smoke tests should not\nitself mutate deployment configuration.\n\nThere are, however, several resources that can be defined as either deployment\nconfiguration or as API resources. For example, shared app domains and isolation\nsegments are both resources that can be created via the API, so it might be\ntempting to have a test create them in a `BeforeSuite`. However, shared app\ndomains and isolation segments really represent deployment configurations.\nAccordingly, smoke tests should not create those resources as part of the test;\ninstead, it should validate (either implicitly or explicitly) that those\nresources have already been created, and configured correctly.\n\nOther API resources, like orgs and spaces that exist simply to be able to push\nan app, can absolutely be created as part of a test.\n\n#### Admin vs. Regular User workflows\nPlease refer to the [Regular vs admin user section](#admin-vs.-regular-user) to\nunderstand the difference between these two workflows.\n\n### Dependency Management\n`cf-smoke-tests` use modules to manage `go` dependencies. These dependencies,\ntogether with the version of `go` itself, are automatically bumped by the CI\npipeline defined in the\n[cf-smoke-tests-release](https://github.com/cloudfoundry/cf-smoke-tests-release/tree/main/ci)\nrepo.\n\nAll `go` packages required to run smoke tests are vendored into the `vendor/`\ndirectory.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Fcf-smoke-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudfoundry%2Fcf-smoke-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudfoundry%2Fcf-smoke-tests/lists"}