{"id":24752021,"url":"https://github.com/inferno-framework/subscriptions-test-kit","last_synced_at":"2025-06-30T12:32:35.495Z","repository":{"id":257813902,"uuid":"869088280","full_name":"inferno-framework/subscriptions-test-kit","owner":"inferno-framework","description":"Inferno Test Kit for FHIR R5-style Subscriptions","archived":false,"fork":false,"pushed_at":"2025-01-22T18:24:18.000Z","size":248,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T19:31:53.885Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/inferno-framework.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.MD","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-07T17:40:36.000Z","updated_at":"2025-01-13T15:49:09.000Z","dependencies_parsed_at":null,"dependency_job_id":"923a6d9d-c5bb-4d1d-b881-55c0cbd1fc64","html_url":"https://github.com/inferno-framework/subscriptions-test-kit","commit_stats":null,"previous_names":["inferno-framework/subscriptions-test-kit"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-framework%2Fsubscriptions-test-kit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-framework%2Fsubscriptions-test-kit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-framework%2Fsubscriptions-test-kit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inferno-framework%2Fsubscriptions-test-kit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/inferno-framework","download_url":"https://codeload.github.com/inferno-framework/subscriptions-test-kit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":236003630,"owners_count":19079666,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-28T10:34:28.818Z","updated_at":"2025-01-28T10:34:29.472Z","avatar_url":"https://github.com/inferno-framework.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FHIR Subscriptions Test Kit\n\nThe Subscriptions Test Kit verifies the conformance of systems to \n[FHIR Subscriptions framework](https://www.hl7.org/fhir/R5/subscriptions.html)\nspecified in the R5 version and above. For R4 and R4B it checks conformance against\nthe STU 1.1 version of the [Subscriptions R5 Backport \nIG](https://hl7.org/fhir/uv/subscriptions-backport/STU1.1/index.html).\nThe test kit includes suites targeting the following actors from the specification:\n\n- **Subscription Servers**: Inferno will simulate a client by requesting a Subscription\n  and accepting and evaluating the resulting notifications from the server under test.\n- **Subscription Clients**: Inferno will simulate a server by waiting for a Subscription\n  request and then sending notifications based on it back to the client under test.\n\nIn each case, content provided by the system under test will be checked individually\nfor conformance and in aggregate to determine that it supports the full set of required\nfeatures.\n\nThis test kit is [open source](#license) and freely available for use or\nadoption by the health IT community including EHR vendors, health app\ndevelopers, and testing labs. It is built using the [Inferno\nFramework](https://inferno-framework.github.io/). The Inferno Framework is\ndesigned for reuse and aims to make it easier to build test kits for any\nFHIR-based data exchange.\n\n## Status\n\nThese tests are a **DRAFT** intended to allow Subscriptions implementers to perform \npreliminary checks of their implementations against Subscription R5 Backport IG's requirements\nand provide feedback on the tests. Future versions of these tests may verify other \nrequirements and may change how these are tested.\n\n## Test Scope and Limitations\n\nThis test is still in draft form and does not test all of the requirements and features\ndescribed in the Subscriptions IG. You can find information on the requirements\nthat the kit covers and does not cover in the [Requirements \nCoverage](lib/subscriptions_test_kit/requirements/generated/subscriptions-test-kit_requirements_coverage.csv)\nCSV document.\n\nSpecific limitations to highlight include\n- Inferno supports only the `rest-hook` channel type. Support for other channels may be added in the future.\n  If there is a channel type that you would like to see verified, please \n  [provide feedback](https://github.com/inferno-framework/subscriptions-test-kit/issues) to that effect.\n- Inferno does not test delivery error handling and recovery scenarios, including\n  the optional `$events` API and event numbering details.\n- Inferno only supports verification of FHIR R4 systems.\n- Inferno does not verify that the shape and content of notifications are appropriate for the triggering\n  Subscription because those details, e.g., the resource types that can be a focus of the notification, \n  are defined within the SubscriptionTopic which is not available in FHIR R4.\n\nSee suite-specific documentation on scope and current limitations\nfor R4 [server](lib/subscriptions_test_kit/docs/subscriptions_r5_backport_r4_server_suite_description.md) and R4\n[client](lib/subscriptions_test_kit/docs/subscriptions_r5_backport_r4_client_suite_description.md)\ntests.\n\n## How to Run\n\nUse either of the following methods to run the suites within this test kit.\nIf you would like to try out the tests but don’t have a Subscriptions implementation, \nthe test home pages include instructions for trying out the tests by running the \nclient and server suites against each other\n\nDetailed instructions can be found in the suite descriptions when the tests\nare run or within this repository for \n[server](lib/subscriptions_test_kit/docs/subscriptions_r5_backport_r4_server_suite_description.md#running-the-tests) and\n[client](lib/subscriptions_test_kit/docs/subscriptions_r5_backport_r4_client_suite_description.md#running-the-tests)\ntests, including [instructions for running the test kits against each \nother](lib/subscriptions_test_kit/docs/subscriptions_r5_backport_r4_client_suite_description.md#sample-execution)\nto see the tests work without bringing an implementation.\n\n### ONC Hosted Instance\n\nYou can run the Subscriptions test kit via the [ONC Inferno](https://inferno.healthit.gov/test-kits/subscriptions/)\nwebsite by choosing the “Subscriptions Test Kit” test kit.\n\n### Local Inferno Instance\n\n- Download the source code from this repository.\n- Open a terminal in the directory containing the downloaded code.\n- In the terminal, run `setup.sh`.\n- In the terminal, run `run.sh`.\n- Use a web browser to navigate to `http://localhost`.\n\nNOTE: running the client and server tests against each other does not work\nwhen running the test kit locally using Docker. If you'd like to use the demo,\nuse the [public instance](https://inferno.healthit.gov/test-kits/subscriptions/)\nor [run locally using Ruby](#development).\n\n## Providing Feedback and Reporting Issues\n\nWe welcome feedback on the tests, including but not limited to the following areas:\n- Verification logic, such as potential bugs, lax checks, and unexpected failures.\n- Requirements coverage, such as requirements that have been missed and tests that necessitate features that the IG does not require.\n- User experience, such as confusing or missing information in the test UI.\n\nPlease report any issues with this set of tests in the issues section of this repository.\n\n## Development\n\nTo make updates and additions to this test kit, see the \n[Inferno Framework Documentation](https://inferno-framework.github.io/docs/),\nparticularly the instructions on \n[development with Ruby](https://inferno-framework.github.io/docs/getting-started/#development-with-ruby).\n\n## Requirements Tracking and Test Coverage\n\nThis test kit includes experimental capabilities to document which tests verify which requirements\nand create a document outlining the coverage of the requirements by the test kit. The functionality\nis currently specific to the Subscriptions test kit and will be generalized and expanded in the future.\n\n### Related Files\n\nThe following files are involved in the requirements tracking and test coverage functionality\nand are all located in the `requirments` folder or sub-folders:\n- [`subscriptions-test-kit_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_requirements.csv): contains\n  the full list of requirements extracted and in scope for this test kit. This can either be created by hand\n  or generated from a set of requirements planning excel documents.\n- [`subscriptions-test-kit_out_of_scope_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_out_of_scope_requirements.csv):\n  contains a list of requirements that will not be tested due to the inability to verify\n  the requirement or other reasons. This can either be created by hand\n  or generated from a set of requirements planning excel documents.\n- [`subscriptions-test-kit_requirements_coverage.csv`](lib/subscriptions_test_kit/requirements/generated/subscriptions-test-kit_requirements_coverage.csv):\n  this generated file contains the list of requirements and which tests in which suites test them. Requirements that\n  are not pertinent to the actor each suite tests are indicated, as are requirements that are explicitly listed in the\n  [`subscriptions-test-kit_out_of_scope_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_out_of_scope_requirements.csv)\n  file.\n\n### Inferno DSL Annotations\n\nThis test kit includes an extension that allows the `verifies_requirements` field \nwithin Inferno Suites, Groups, and Tests to be populated with a comma-seperated\nlist of requirement identifiers that correspond to rows in the \n[`subscriptions-test-kit_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_requirements.csv) file, either in the form `'\u003crequirement set id\u003e@\u003crequiment id\u003e'`.\n\nIncluding a requirement within the `verifies_requirements` list asserts that execution of the test, group, or suite\nverifies the referenced requirement and a pass indicates that the system meets the requirement. At this time,\nthere is no way to indicate that the verification is partial.\n\n### Rake Tasks\n\nTwo rake tasks can be used to generate the files described above:\n- `bundle exec rake \"requirements:collect[\u003cpath to directory with planning excel file(s)\u003e]\"`:\n  This task converts one or more requirements planning files into the\n  [`subscriptions-test-kit_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_requirements.csv) and\n  [`subscriptions-test-kit_out_of_scope_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_out_of_scope_requirements.csv)\n  files.\n    - NOTE: an issue with this task can cause the links the the `URL` column to be truncated in\n      a valid, but not ideal way. check for changes of this nature before committing updates\n      to the [`subscriptions-test-kit_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_requirements.csv) file. To avoid the issue, make sure that in the \n      source excel files the `URL` column has no hyperlinks by selecting the column, right clicking,\n      and selecting \"Remove Hyperlinks\". If that menu option is not present, there are no hyperlinks\n      present. Once you have confirmed that no hyperlinks are present, run the collect script.\n- `bundle exec rake requirements:generate_coverage`: This task uses the information in the\n  [`subscriptions-test-kit_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_requirements.csv) file, the\n  [`subscriptions-test-kit_out_of_scope_requirements.csv`](lib/subscriptions_test_kit/requirements/subscriptions-test-kit_out_of_scope_requirements.csv)\n  file, and the `verifies_requirements` annotations in the test kit to create the\n  [`subscriptions-test-kit_requirements_coverage.csv`](lib/subscriptions_test_kit/requirements/generated/subscriptions-test-kit_requirements_coverage.csv) file.\n\nEach has a corresponding \"check\" rake task (`requirements:check_collection` and \n`\"requirements:check_coverage[\u003cpath to directory with planning excel file(s)\u003e]\"`) that\ncan be built into commit pipelines to ensure that the files are in sync with the other content.\n\n## License\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use\nthis file except in compliance with the License. You may obtain a copy of the\nLicense at\n```\nhttp://www.apache.org/licenses/LICENSE-2.0\n```\nUnless required by applicable law or agreed to in writing, software distributed\nunder the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR\nCONDITIONS OF ANY KIND, either express or implied. See the License for the\nspecific language governing permissions and limitations under the License.\n\n## Trademark Notice\n\nHL7, FHIR and the FHIR [FLAME DESIGN] are the registered trademarks of Health\nLevel Seven International and their use does not constitute endorsement by HL7.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finferno-framework%2Fsubscriptions-test-kit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finferno-framework%2Fsubscriptions-test-kit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finferno-framework%2Fsubscriptions-test-kit/lists"}