{"id":26514858,"url":"https://github.com/devcyclehq/test-harness","last_synced_at":"2025-07-24T10:06:34.886Z","repository":{"id":63382167,"uuid":"566947962","full_name":"DevCycleHQ/test-harness","owner":"DevCycleHQ","description":"DevCycle - Test Harness","archived":false,"fork":false,"pushed_at":"2025-07-23T18:28:41.000Z","size":18990,"stargazers_count":5,"open_issues_count":2,"forks_count":1,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-07-23T21:06:51.745Z","etag":null,"topics":["continuous-delivery","continuous-deployment","devcycle","devops","feature-flags","feature-toggles","openfeature"],"latest_commit_sha":null,"homepage":"https://docs.devcycle.com/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DevCycleHQ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-11-16T18:49:28.000Z","updated_at":"2025-07-23T18:28:45.000Z","dependencies_parsed_at":"2024-01-09T21:23:45.999Z","dependency_job_id":"c9d68109-2f63-4ac5-9a1a-4a2d2f6cf1dc","html_url":"https://github.com/DevCycleHQ/test-harness","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/DevCycleHQ/test-harness","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevCycleHQ%2Ftest-harness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevCycleHQ%2Ftest-harness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevCycleHQ%2Ftest-harness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevCycleHQ%2Ftest-harness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DevCycleHQ","download_url":"https://codeload.github.com/DevCycleHQ/test-harness/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DevCycleHQ%2Ftest-harness/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266823623,"owners_count":23990044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-delivery","continuous-deployment","devcycle","devops","feature-flags","feature-toggles","openfeature"],"created_at":"2025-03-21T05:29:34.430Z","updated_at":"2025-07-24T10:06:34.881Z","avatar_url":"https://github.com/DevCycleHQ.png","language":"TypeScript","readme":"# SDK Test Harness\n\nThe shared test harness for testing DevCycle SDKs\n\n## Introduction\n\nThe test harness is a tool which defines a set of standardized tests that can run against multiple SDKs written in\ndifferent languages.\n\nCurrently it includes tests for serverside SDKs, for both cloud and local bucketing modes.\n\nThe tests are written in Jest and run against a series of \"proxy servers\" written in the native language of each SDK.\n\n## Running the Tests\n\n### Prerequisites\n\nYou must have Docker installed and running on your machine. You must also have a working copy of Node 18+ and Yarn.\n\n### Steps\n\n-   Install dependencies with `yarn`\n-   Run test suite with `yarn test`\n\nThis will automatically start the proxy servers using `docker-compose` and run the tests against them.\n\n### Environment Variables\n\nThere are environment variables which can be used to control which SDK to run the tests for, and which version\nThey are:\n\n-   `SDKS_TO_TEST` - The name of the SDK to run the tests for. Possible values include `nodejs`, `go` etc.\n-   `{SDK NAME}_SDK_VERSION` - The version of the given SDK to run the tests for. eg. `NODEJS_SDK_VERSION=1.2.3`\n-   `SDK_GITHUB_SHA` - The sha of an unreleased commit of the SDK to run the tests against. This version will be checked\n    out directly from Github.\n\n## Debugging a failing test\n\nIf a test run fails for a particular SDK change, use the environment variables above to run the tests locally\nagainst the version of the SDK causing the issue. It also helps to isolate the test failure by running against\na particular test file, or using `it.only` to run only a single test within a file.\n\nYou can run yarn much faster against a single test file with:\n\n```\nyarn test --testPathPattern='variable.local'\n```\n\nor against a single test in that file (the matching against dynamic test names is tricky, so you might not be able to just run one):\n\n```\nyarn test --testPathPattern='variable.local' --testNamePattern='should return a valid unicode string'\n```\n\nIt is also typically necessary to run the SDK proxy server locally in order to log / debug what is going on.\nSee the section below on running the harness against a local SDK.\n\nIf debugging tests against a known version, you can speed up iteration by running with the watch flag:\n\n```\nyarn test --watch\n```\n\nThat will keep the docker containers running the proxies alive, and pop up a menu in the terminal for re-running tests, failures, etc.\n\n## Running the harness against a local SDK\n\nYou can run the proxy server for the SDK you want to work on using:\n`yarn start:{sdk}` eg. `yarn start:nodejs`\n\nWhen running proxy tests, use the following environment variables to specify local mode:\n\n```\nLOCAL_MODE=1 SDKS_TO_TEST=nodejs yarn test\n```\n\nThis will point the test harness at a proxy running at http://localhost:3000, and disable all the containers in docker-compose. See [these instructions](docs/LOCAL.md) for details on how to run the proxy for each SDK.\n\nIf you'd like to optionally target local SDK versions, see the relevant sections in `docs/LOCAL.md` for exact examples and steps for each proxy.\n\nA specific version of docker-compose may be required to support the profiles in the docker-compose.yml config file. 1.29.2 seems to work.\n\nYou may still need to run the package manager for the particular proxy (e.g. `yarn`, `go mod tidy` etc.)\n\n## Development\n\nTo understand how the system works and write new proxies, see the documentation in the `docs` folder.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevcyclehq%2Ftest-harness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdevcyclehq%2Ftest-harness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdevcyclehq%2Ftest-harness/lists"}