{"id":23421098,"url":"https://github.com/nhsdigital/healthcare-worker-api","last_synced_at":"2025-04-09T09:34:25.602Z","repository":{"id":261051292,"uuid":"861838733","full_name":"NHSDigital/healthcare-worker-api","owner":"NHSDigital","description":"Healthcare worker API repository","archived":false,"fork":false,"pushed_at":"2025-04-02T20:14:51.000Z","size":1583,"stargazers_count":0,"open_issues_count":11,"forks_count":0,"subscribers_count":1,"default_branch":"develop","last_synced_at":"2025-04-02T20:36:21.252Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NHSDigital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":".github/SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-23T15:40:46.000Z","updated_at":"2025-03-17T10:04:43.000Z","dependencies_parsed_at":"2024-11-04T13:17:44.368Z","dependency_job_id":"63b43292-6914-48b8-91ef-c90e274ed4f0","html_url":"https://github.com/NHSDigital/healthcare-worker-api","commit_stats":null,"previous_names":["nhsdigital/healthcare-worker-api"],"tags_count":1,"template":false,"template_full_name":"NHSDigital/repository-template","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fhealthcare-worker-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fhealthcare-worker-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fhealthcare-worker-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fhealthcare-worker-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NHSDigital","download_url":"https://codeload.github.com/NHSDigital/healthcare-worker-api/tar.gz/refs/heads/develop","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248012988,"owners_count":21033283,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T02:14:01.923Z","updated_at":"2025-04-09T09:34:25.585Z","avatar_url":"https://github.com/NHSDigital.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Healthcare Worker API\n\nThe Healthcare Worker API is a Python app deployed to AWS.\n\n## Setup\n\nBefore performing any local development we need to perform some basic setup tasks.\n\n### Python\n\nThis is a Python API with dependencies pulled in using poetry. In order to run locally you must have [Python](https://www.python.org/downloads/)\nand [poetry](https://python-poetry.org/docs/) installed.\n\nPyCharm is the recommended IDE for this project. When setting up for the first time we need to configure the Python\ninterpreter, which will also set up a virtual environment for our dependency installs. To set this up:\n\n1. Open one of the Python files (under `src`) in PyCharm\n2. Click the \"Configure Python Interpreter\" link in the top of the window\n3. Select \"Add New Interpreter\" -\u003e \"Add Local Interpreter\"\n4. Leave the directory as the default (should be `venv` within the root of the project)\n5. Ensure that the Python version is set to at Python 3.12\n6. Check that the `venv` directory has been created and that the missing interpreter warning no longer displays\n\nIf you want to install/run from a terminal you will need to activate the venv in that terminal. The command for this\nvaries slightly based on OS.\n\nMacOS: `source venv/bin/activate`\nWindows: `.\\venv\\Scripts\\activate.bat`\n\nOnce you've switched to the venv you can install dependencies with `poetry install`.\n\nWe can run the application locally with the command `poetry run start`\n\nIf you need to manually deploy your local app to an environment then you need to build it first. Run the `./scripts/build-app.sh` script\nfrom the root directory to generate the zip file that needs to be uploaded. Then run `./scripts/deploy-app.sh` to publish to app\nto the S3 artifact bucket.\n\n### Terraform\n\nThe `infrastructure` directory contains everything needed to define an HCW AWS environment. Generally these changes should\nbe deployed out through our GitHub pipelines, but sometimes you may need to test / build / deploy locally. This section\nguides through how to do that.\n\n1. Install and set up the [proxygen-cli](https://pypi.org/project/proxygen-cli/). This allows us to deploy new APIM apps for our PRs and static environments.\n2. Install and setup [yq](https://github.com/mikefarah/yq), which is used to make minor changes to the specification yaml in order to support API deployments (e.g. renaming the title to include the PR number)\n3. Install the [Terraform CLI](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) if you haven't already\n4. Install the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) if you haven't already\n5. Save your AWS credentials\n   1. Go to the [AWS account list](https://d-9c67018f89.awsapps.com/start/#/?tab=accounts) page in a browser\n   2. Select the environment you want to deploy to\n   3. Click on the \"Access Keys\" link\n   4. Copy the environment variables and paste into your terminal. This should have set `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY` and `AWS_SESSION_TOKEN`.\n6. Go to the `instrastructure` directory\n7. Run `terraform init`. You should see a message including the message \"Terraform has been successfully initialized!\"\n8. Switch to the Terraform workspace you want with `terraform workspace select \u003cworkspace\u003e`. See below for workspace details\n   1. If this is the first deployment to this workspace then you will need to run `terraform workspace new \u003cworkspace\u003e` first\n9. Run `terraform plan -var-file=environments/dev.tfvars` to validate your changes and see what impact it will have if deployed\n    1. This is important. **Make sure the plan represents the change you want to make before running the apply command**\n10. If you're happy with the above plan, run `terraform apply -var-file=environments/dev.tfvars` to make the change in AWS\n    1. If you're deploying to an app environment (i.e. not management) then you'll also need to specify location of the S3 lambda code in S3. For example, `-var \"app_s3_filename=66374856c6c908c50e5d0974704b0e727106a934.zip\"`. Since you need a valid zip file before deployments, it's almost always easier to let the update happen automatically through the PR.\n\nA Terraform linter runs on each push to a PR, the command `terraform fmt -recursive` will resolve any simple formatting issues for fix that status failure.\n\n#### Terraform Workspaces\n\nWe make use of Terraform workspaces to keep environments as separate as possible, but there will always be some resources\nwhich need to be shared between environments. Most of these are in the dedicated \"management\" account, but some need to\nbe in the relevant AWS account (e.g. deployment roles). The following table shows the mapping between AWS account,\nTerraform workspace, and Terraform environments file.\n\nIn most cases this is detail you don't need to worry about, but it's worth noting that management changes are not\ndeployed until you merge to develop.\n\n| AWS Account               | Terraform Workspace | Environments File | Notes                                                                                                                                               |\n|---------------------------|---------------------|-------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------|\n| management (209479271736) | management          | mgmt.tfvars       | Anything common between all environments (e.g. VPN connection). Deployed on merge to develop.                                                       |\n| dev  (535002889321)       | mgmt                | dev.tfvars        | Resources shared between environments, but specific to the account (e.g. app deployment role). Deployed on merge to develop.                        |\n| dev  (535002889321)       | pr-*                | dev.tfvars        | App deployment of lambda and associated resources. Deployed on push to an open PR, contains latest app code from the relevant branch.               |\n| dev  (535002889321)       | ft                  | dev.tfvars        | App deployment of lambda and associated resources. Deployed on merge to develop.                                                                    |\n| int  (711387117641)       | int                 | int.tfvars        | App deployment of lambda and associated resources. Deployed on run from static env deployment pipeline.                                             |\n| int  (711387117641)       | mgmt-int            | int.tfvars        | Resources shared between environments, but specific to the account (e.g. app deployment role). Deployed on run from static env deployment pipeline. |\n| prod (266735814611)       | prod                | prod.tfvars       | App deployment of lambda and associated resource for production.                                                                                    |\n| prod (266735814611)       | mgmt-prod           | prod.tfvars       | Deployment resources for production environment                                                                                                     |\n\n## Environments \u0026 Pipelines\n\nWe have a number of dev environments and static environments for more formal testing. The following is our current environments, along with their AWS account and their general purpose:\n\n* Dev environments - dev - created automatically with each PR\n* FT - dev - created automatically from the latest code on the develop branch\n* Sand - int - for supplier testing with minimal barriers, designed to return a representative response but not a true integration\n* Int - int - for integration testing with suppliers\n* Ref - int - for formal release testing before deploying to production\n* Prod - prod - production environment\n\n### Development process\n\n#### Code Structure\n\nThis repo contains all the source code for the project. This includes the production code (in `src`), integration tests (`integration_tests`),\nNFT tests (`nft`), Terraform (`infrastructure`) and the [publicly deployed specification](https://digital.nhs.uk/developer/api-catalogue/healthcare-fhir-api/content) (`specification`).\n\nThe production code is split into two main layers. The requests from the API Gateway are routed to the appropriate handler in `request_handlers`\nbased on the called endpoint. These handlers take data from the appropriate data source and convert into the FHIR format (all FHIR classes are in their own `fhir` folder).\nThe handler then returns the FHIR which is returned to callers of the API.\n\nThis app supports the use of FHIR's `_include` and `_revinclude` query parameters. These query parameters allow API consumers to request more than the base resource that's being searched for.\n`_include` looks for other objects referenced by any in the response, `_revinclude` looks for any other objects which reference any already in the response.\nThe functionality in `include_populator.py` supports this functionality by checking the responses from the data layer. This is applied automatically\nto all handlers so long as the data layer returns the appropriate information, and it's mapped into FhirReference objects by the handler. See the `FhirPractitioner` and\n`FhirPractitionerRole` classes for examples of how this works.\n\n#### PRs\n\n**All commits need to be [signed](https://docs.github.com/en/authentication/managing-commit-signature-verification/signing-commits)** else the PR will be automatically rejected.\n\nAll work should be performed against a ticket in the HCW jira project. The changes should be made on a branch specific to that ticket.\nWe don't have any formal branch naming conventions, but as a minimum the ticket number must be on the branch.\nThere is no restriction of commit names on the branch, but all PRs must be squashed when merging.\nThe squashed commit message should start with the jira ticket number and include a brief description of the change, (e.g. \"HCW-76: Deployment Pipeline\").\n\nCreating a PR will automatically trigger a few different processes. The PR itself shows the status of a number of checks performed on the code.\nThis includes things like linting, Terraform format checks and spelling checker. It also automatically triggers the\ndeployment of the dev environment based on the PR. This shows on the PR as the \"HCW Deployment\" status check which appears on each commit shortly after being pushed.\nIt will be marked as \"pending\" until there's either an error on the pipeline or the integration tests successfully complete with all tests passing.\n\nYou can check on the status of your build and deployment through the AWS console in the dev account (note that this may change to the management account under [HCW-100](https://nhsd-jira.digital.nhs.uk/browse/HCW-100)).\nThe hcw-api-deployment pipeline will trigger within a minute of the PR creation (or new commit to an existing PR). The history page shows current and previous runs - see [AWS pipeline execution history page](https://eu-west-2.console.aws.amazon.com/codesuite/codepipeline/pipelines/hcw-api-deployment/executions) (you can use the \"Source revisions\" column to make sure you've found your build)\n\nEach pipeline starts with the \"build\" which performs a poetry build to generate the files that will deployed to the lambda. The S3-Upload action then zips and uploads the files to S3, this ensures that future deployments will be deploying exactly the same code.\nThe \"Deploy\" action performs any relevant infrastructure changes, including updating the application lambda to the latest code. The dev environment is up to date once this step completes.\n\n*Note that not all of the Terraform in the repository is applied at this stage. There are some resources which are common between environments, they are only updated once the PR is merged into develop. See above Terraform section for more information.*\n\n#### Post Merge Process\n\nThe same pipeline (hcw-api-deployment) is triggered for merges to develop, but it deploys to \"FT\" instead of a PR dev environment. Once the deployment is complete it also triggers the \"hcw-api-static-env-deployment\" job.\nThe main difference is that this pipeline requires approval before every deployment, ensuring that we don't update a higher environment accidentally.\nThe deployments happen in other environments, so you'll need to log into the int or prod AWS accounts to see their logs, but the pipeline will show if the job ran successfully or not.\n\n## Testing\n\nThe best way to test an environment is to use the integration tests in this repository. If you want to test against a branch\nthen you need to create a PR first. This deploys the application to AWS and APIM, which is where the integration tests run against.\nThe following steps describe how to set up and run these integration tests against any environment:\n\n1. Modify the file at `integration_tests/locals.properties` based on the environment you're testing\n   1. If you're testing a PR environment then you need to populate the `env` (e.g. `pr-16`) and `client_id` values. The `client_id` can be found in the deploy job output (`Client id = \u003cclient_id\u003e`)\n   2. If you're testing a static environment (e.g. ft) then you only need to put the environment name into the `env` field\n2. From the repository root make sure you've run a `poetry install` for any dependencies needed by the tests\n3. You can now run all integration tests from the command line by going to the `integration_tests` directory and running `pytest`\n   1. Note that running from the repository root does not trigger the integration tests, this is to separate them from the unit tests for normal running\n   2. You can also run individual tests from inside IDEs like PyCharm\n\n### Manual testing\n\nThe current pipeline will automatically create APIM apps, and the integration tests handle authentication automatically.\nBefore this was available we had to go through those steps manually. While this shouldn't be necessary now, it's useful to\nkeep the process documented. This section lists the steps required to create an APIM app and send manual requests through\npostman. It assumes that there is already an environment in AWS to point to, and an API product in APIM.\n\n1. Before starting, check the PR number of your raised PR. This is the number at the end of the PR URL, it also displays in the title after the #.\n2. Connect to the HSCN VPN\n3. Go to `https://dos-dev.ptl.api.platform.nhs.uk/` and login. You can create a dev account through the UI if you haven't already.\n4. Click on \"Environment Access\"\n5. Click \"Add new application\"\n6. Select \"Development\"\n7. Select \"Me\"\n8. Enter an application name like \"HCW PR-\u003cpr_number\u003e\" and click \"Continue\"\n9. Select \"Create Application\"\n10. Select \"View your new application\"\n11. On the \"Public key URL\" line click \"Edit\"\n12. Enter the URL of `https://raw.githubusercontent.com/NHSDigital/identity-service-jwks/refs/heads/main/jwks/internal-dev/5eef95c7-031c-4d7b-ab58-1fee6e91a915.json`, this related to a known key pair so we can generate valid requests using it.\n13. Select \"Save\" and then once confirmed click on your app name in the top breadcrumbs to return to the previous page\n14. Select \"Add APIs\"\n15. Search for PR-\u003cpr_number\u003e to find your app instance. Note that there are other projects in this space, so make sure you've selected a healthcare worker API\n16. Select your PR and click \"Save\"\n17. On the \"Active API keys\" line click \"Edit\"\n18. Make note of the key shown on this page as it's needed to generate valid requests\n\nWith the above steps you have created a valid APIM app which will route requests to your PR. We can now start sending\nrequests through to the HCW APIs. In order for these requests to be successful we need to authenticate with APIM using\nan access token. This repository includes a script for generating a valid access token based on the above keypair.\n\n1. Make sure that you have the private key at `integration_tests/utils/test-1.pem`. This file is not checked into git for security. It can be downloaded from AWS Secret Manager `internal-dev/request-key` secret.\n2. Install the poetry dependencies from the top level if you haven't already: `poetry install`\n3. Run the script with the following command, replacing `\u003capi_key\u003e` with the API key from your app: `poetry run token \u003capi_key\u003e`\n4. The script will output the access token. This needs to be included in any requests in the `Authorization` header as `Bearer \u003caccess_token\u003e`\n\n## General Maintenance\n\n### Public App Documentation\n\nThe [public app documentation](https://digital.nhs.uk/developer/api-catalogue/healthcare-worker) is taken from the OAS\nfile at specification/healthcare-worker-api.yaml. Merging to develop will publish the specification to the\n[UAT catalogue](https://uat2.nhsd.io/developer/api-catalogue/healthcare-worker), and deploying to INT will deploy to the\nmain public catalogue.\n\n### Pipeline tokens\n\nWe have a few tokens saved as secrets in AWS so that we can publish the state of deployments. We currently have GitHub and slack tokens.\nThe GitHub token is for publishing the state of PR deployment to the relevant commit. The slack token is used for publishing\nthe result of deployments to FT into a dedicated Slack channel.\n\nThese tokens could expire, or otherwise need replacing when we notice a problem. The following sections describe what steps need to be taken to refresh them.\n\n### Renewing GitHub token\n\nThe [GitHub documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens)\nexplains how to create a new fine-grained access token. It only needs access to this repository and needs the \"Read access to metadata\"\nand \"Read and Write access to commit statuses\" permissions.\n\nOnce the new token has been created, the value needs to be saved in the `github-access-token` secret (in secrets manager) in the management AWS account.\n\n### Renewing Slack token\n\nThe Slack token is based on the bot app we created specifically for HCW deployments. You can manage this app from the\n[Slack apps page](https://api.slack.com/apps). From the app config go to \"OAuth \u0026 Permissions\", the OAuth token should\nbe displayed under \"OAuth Tokens\".\n\nSave this token as the `slack-access-token` secret (in secrets manager) in the management AWS account.\n\nIf the bot app ever needs creating then it simply needs the `chat:write` permission, and then for the bot user to be\nadded to the appropriate Slack channel. Make sure to set the value of `SLACK_CHANNEL_ID` in `status_reporting.py` appropriately.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fhealthcare-worker-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnhsdigital%2Fhealthcare-worker-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fhealthcare-worker-api/lists"}