{"id":23421262,"url":"https://github.com/nhsdigital/immunisation-fhir-api","last_synced_at":"2026-04-01T18:51:46.223Z","repository":{"id":147008150,"uuid":"615276722","full_name":"NHSDigital/immunisation-fhir-api","owner":"NHSDigital","description":null,"archived":false,"fork":false,"pushed_at":"2026-01-29T03:14:43.000Z","size":29480,"stargazers_count":2,"open_issues_count":9,"forks_count":3,"subscribers_count":23,"default_branch":"master","last_synced_at":"2026-01-29T03:24:06.860Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NHSDigital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-03-17T10:37:24.000Z","updated_at":"2026-01-28T12:04:39.000Z","dependencies_parsed_at":"2024-02-05T09:32:33.325Z","dependency_job_id":"8a9708cc-966d-412e-b1e6-5eff354585fd","html_url":"https://github.com/NHSDigital/immunisation-fhir-api","commit_stats":null,"previous_names":[],"tags_count":674,"template":false,"template_full_name":null,"purl":"pkg:github/NHSDigital/immunisation-fhir-api","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fimmunisation-fhir-api","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fimmunisation-fhir-api/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fimmunisation-fhir-api/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fimmunisation-fhir-api/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NHSDigital","download_url":"https://codeload.github.com/NHSDigital/immunisation-fhir-api/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fimmunisation-fhir-api/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29047789,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-03T14:55:20.264Z","status":"ssl_error","status_checked_at":"2026-02-03T14:55:19.725Z","response_time":96,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-23T02:14:36.050Z","updated_at":"2026-04-01T18:51:46.214Z","avatar_url":"https://github.com/NHSDigital.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# immunisation-fhir-api\n\nSee `README.specification.md` for details of the API specification's development.\n\n## Spelling\n\nRefer to the FHIR Immunization resource capitalised and with a \"z\" as FHIR is U.S. English.\n\nAll other uses are as British English i.e. \"immunisation\".\n\nSee https://nhsd-confluence.digital.nhs.uk/display/APM/Glossary.\n\n## Directories\n\n**Note:** Each Lambda has its own `README.md` file for detailed documentation. For non-Lambda-specific folders, refer to `README.specification.md`.\n\n### Lambdas (compute microservices)\n\n| Folder                   | Description                                                                                                                                                                             |\n| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |\n| `ack_backend`            | **Imms Batch** – Generates the final Business Acknowledgment (BUSACK) file from processed messages and writes it to the designated destination.                                         |\n| `backend`                | **Imms API** – Handles CRUD operations for the Immunisation API.                                                                                                                        |\n| `batch_processor_filter` | **Imms Batch** – Controller function that consumes from a queue and forwards file event for processing if filter conditions are met.                                                    |\n| `delta_backend`          | **Imms Sync** – Lambda function that reacts to events in the Immunisation Event Data Store (IEDS).                                                                                      |\n| `filenameprocessor`      | **Imms Batch** – Validates and processes new batch file events.                                                                                                                         |\n| `id_sync`                | **Imms Cross-cutting** – Handles [MNS](https://digital.nhs.uk/developer/api-catalogue/multicast-notification-service) NHS Number Change events and applies updates to affected records. |\n| `mesh_processor`         | **Imms Batch** – Triggered when new files are received via MESH. Moves them into the Imms Batch processing system.                                                                      |\n| `mns_subscription`       | **Imms Cross-cutting** – Simple helper Lambda which sets up our required MNS subscription. Used in pipelines in DEV.                                                                    |\n| `perf_tests`             | **Imms API** – Locust performance tests for the Immunisation API.                                                                                                                       |\n| `recordforwarder`        | **Imms Batch** – Consumes from the stream and applies the processed batch file row operations (CUD) to IEDS.                                                                            |\n| `recordprocessor`        | **Imms Batch** – ECS Task - **not** a Lambda function - responsible for processing batch file rows and forwarding to the stream.                                                        |\n| `redis_sync`             | **Imms Cross-cutting** – Handles config file updates. E.g. disease mapping or permission files.                                                                                         |\n| `shared`                 | **Imms Cross-cutting** – Shared `common` code that can be shared by other Lambda functions. This is not a standalone Lambda.                                                            |\n\n---\n\n### Pipelines\n\nDue to the timing of the project's inception, Azure pipelines have been inherited from the API Management team for deploying\nthe Apigee proxy and sandbox. The new way to manage and deploy said resources is [Proxygen](https://digital.nhs.uk/developer/api-catalogue/proxy-generator).\n\nIn future a migration plan will be provided by the API Management team so we can move to the new process and use purely\nGitHub Actions for our entire pipeline.\n\n| Folder    | Description                                                                                                                     |\n| --------- | ------------------------------------------------------------------------------------------------------------------------------- |\n| `azure`   | Pipeline definition and orchestration code concerned purely with the management and deployment of the Apigee proxy and sandbox. |\n| `.github` | Pipeline definition and orchestration code for deploying and testing our backend project.                                       |\n\n---\n\n### Infrastructure\n\n| Folder                 | Description                                                     |\n| ---------------------- | --------------------------------------------------------------- |\n| `account`              | Base infrastructure components deployed on a per account basis. |\n| `instance`             | Core Terraform app infrastructure.                              |\n| `terraform_aws_backup` | Streamlined backup processing with AWS.                         |\n| `proxies`              | Apigee API proxy definitions.                                   |\n\n---\n\n### Tests\n\n| Folder           | Description                                                                   |\n| ---------------- | ----------------------------------------------------------------------------- |\n| `e2e_automation` | End-to-end tests executed during PR pipelines using the pytest-bdd framework. |\n\n---\n\n### Utilities\n\n| Folder              | Description                                                   |\n| ------------------- | ------------------------------------------------------------- |\n| `quality_checks`    | Dependencies for linting and formatting Python code           |\n| `utilities/scripts` | Standalone or reusable scripts for development and automation |\n| `specification`     | Specification files to document API and related definitions   |\n| `sandbox`           | Simple sandbox API                                            |\n\n---\n\n## Background: Python Package Management and Virtual Environments\n\n- `pyenv` manages multiple Python versions at the system level, allowing you to install and switch between different Python versions for different projects.\n- `direnv` automates the loading of environment variables and can auto-activate virtual environments (.venv) when entering a project directory, making workflows smoother.\n- `.venv` (created via python -m venv or poetry) is Python’s built-in tool for isolating dependencies per project, ensuring that packages don’t interfere with global Python packages.\n- `Poetry` is an all-in-one dependency and virtual environment manager that automatically creates a virtual environment (.venv), manages package installations, and locks dependencies (poetry.lock) for reproducibility, making it superior to using pip manually and it is used in all the lambda projects.\n\n## Project structure\n\nTo support a modular and maintainable architecture, each Lambda function in this project is structured as a self-contained folder with its own dependencies, configuration, and environment.\n\nWe use Poetry to manage dependencies and virtual environments, with the virtualenvs.in-project setting enabled to ensure each Lambda has an isolated `.venv` created within its folder.\n\nAdditionally, direnv is used alongside `.envrc` and `.env` files to automatically activate the appropriate virtual environment and load environment-specific variables when entering a folder.\n\nEach Lambda folder includes its own `.env` file for Lambda-specific settings, while the project root contains a separate `.env` and `.venv` for managing shared tooling, scripts, or infrastructure-related configurations. This setup promotes clear separation of concerns, reproducibility across environments, and simplifies local development, testing, and packaging for deployment.\n\n## Environment setup\n\nThese dependencies are required for running and debugging the Lambda functions and end-to-end (E2E) tests.\n\n### Install dependencies\n\nSteps:\n\n1. Install [WSL](https://learn.microsoft.com/en-us/windows/wsl/install) if running on Windows and install [Docker](https://docs.docker.com/engine/install/).\n2. Install the following tools inside WSL. These will be used by the lambda and infrastructure code:\n\n- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)\n- [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)\n\n3. Open VS Code and click the bottom-left corner (blue section), then select **\"Connect to WSL\"** and choose your WSL distro (e.g., `Ubuntu-24.04`).\n   Once connected, you should see the path as something similar to: `/mnt/d/Source/immunisation-fhir-api/backend`.\n4. Run the following commands to install dependencies\n\n    ```\n    sudo apt update \u0026\u0026 sudo apt upgrade -y\n    sudo apt install -y make build-essential libssl-dev zlib1g-dev \\\n        libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \\\n        libncurses5-dev libncursesw5-dev xz-utils tk-dev libffi-dev \\\n        liblzma-dev git libgdbm-dev libgdbm-compat-dev\n    pip install --upgrade pip\n    ```\n\n5. Configure pyenv.\n\n    ```\n    pyenv install --list | grep \"3.11\"\n    pyenv install 3.11.13 #current latest\n    ```\n\n6. Install direnv if not already present, and hook it to the shell.\n\n    ```\n    sudo apt-get update \u0026\u0026 sudo apt-get install direnv\n    echo 'eval \"$(direnv hook bash)\"' \u003e\u003e ~/.bashrc\n    ```\n\n7. Install poetry\n\n    ```\n    pip install poetry\n    ```\n\n8. Install pre-commit hooks. Ensure you have installed nodejs at the same version or later as per .tool-versions and\n   then, from the repo root, run:\n    ```\n    npm install\n    ```\n\n### Setting up a virtual environment with poetry\n\nThe steps below must be performed in each Lambda function folder and e2e_automation folder to ensure the environment is correctly configured.\n\nFor detailed instructions on running individual Lambdas, refer to the README.md files located inside each respective Lambda folder.\n\nSteps:\n\n1. Set the python version in the folder with the code used by lambda for example `./backend` (see [lambdas](#lambdas)) folder.\n\n    ```\n    pyenv local 3.11.13 # Set version in backend (this creates a .python-version file)\n    ```\n\n    Note: consult the lambda's `pyproject.toml` file to get the required Python version for this lambda. At the time of writing, this is `~3.10` for the batch lambdas and `~3.11` for all the others.\n\n2. Configure poetry\n\n    ```\n    ### Point poetry virtual environment to .venv\n    poetry config virtualenvs.in-project true\n    poetry env use $(pyenv which python)\n    poetry env info\n    ```\n\n3. Create an .env file and add environment variables.\n\n    ```\n    AWS_PROFILE={your_profile}\n    IMMUNIZATION_ENV=local\n    ```\n\n    For unit tests to run successfully, you may also need to add an environment variable for PYTHONPATH. This should be:\n\n    ```\n    PYTHONPATH=src:tests\n    ```\n\n4. Configure `direnv` by creating a `.envrc` file in the backend folder. This points direnv to the `.venv` created by poetry and loads env variables specified in the `.env` file\n\n    ```\n    export VIRTUAL_ENV=\".venv\"\n    PATH_add \"$VIRTUAL_ENV/bin\"\n\n    dotenv\n    ```\n\n5. Restart bash and run `direnv allow`. You should see something similar like:\n    ```\n    direnv: loading /mnt/d/Source/immunisation-fhir-api/.envrc\n    direnv: export +AWS_PROFILE +IMMUNIZATION_ENV +VIRTUAL_ENV ~PATH\n    ```\n    Test if environment variables have been loaded into shell: `echo $IMMUNIZATION_ENV`.\n\n#### Running Unit Tests from the Command Line\n\nIt is not necessary to activate the virtual environment (using `source .venv/bin/activate`) before running a unit test suite from the command line; `direnv` will pick up the correct configurations for us. Run `pip list` to verify that the expected packages are installed. You should for example see that `recordprocessor` is specifically running `moto` v4, regardless of which if any `.venv` is active.\n\n### Setting up the root level environment\n\nThe root-level virtual environment is primarily used for linting, as we create separate virtual environments for each folder that contains Lambda functions.\nSteps:\n\n1. Follow instructions above to [install dependencies](#install-dependencies) \u0026 [set up a virtual environment](#setting-up-a-virtual-environment-with-poetry).\n   **Note: While this project uses Python 3.11 (e.g. for Lambdas), the NHSDigital/api-management-utils repository — which orchestrates setup and linting — defaults to Python 3.8.\n   The linting command is executed from within that repo but calls the Makefile in this project, so be aware of potential Python version mismatches when running or debugging locally or in the pipeline.**\n2. Run `make lint`. This will:\n    - Check the linting of the API specification yaml.\n    - Run Flake8 on all Python files in the repository, excluding files inside .venv and .terraform directories.\n\n## IDE setup\n\nThe current team uses VS Code mainly. So this setup is targeted towards VS code. If you use another IDE please add the documentation to set up workspaces here.\n\n### VS Code\n\nThe project must be opened as a multi-root workspace for VS Code to know that specific lambdas (e.g. `backend`) have their own environment.\nThis example is for `backend`; substitute another lambda name in where applicable.\n\n- Open the workspace `immunisation-fhir-api.code-workspace`.\n- Copy `backend/.vscode/settings.json.default` to `backend/.vscode/settings.json`, or merge the contents with\n  your existing file.\n- Similarly, copy or merge `backend/.vscode/launch.json.default` to `backend/.vscode/launch.json`.\n\nVS Code will automatically use the `backend` environment when you're editing a file under `backend`.\n\nDepending on your existing setup VS Code might automatically choose the wrong virtualenvs. Change it\nwith `Python: Select Interpreter`.\n\nThe root (`immunisation-fhir-api`) should point to the root `.venv`, e.g. `/mnt/d/Source/immunisation-fhir-api/.venv/bin/python`.\n\nMeanwhile, `backend` should be pointing at (e.g.) `/mnt/d/Source/immunisation-fhir-api/backend/.venv/bin/python`\n\n#### Running Unit Tests\n\nNote that unit tests can be run from the command line without VSCode configuration.\n\nIn order that VSCode can resolve modules in unit tests, it needs the PYTHONPATH. This should be setup in `backend/.vscode/launch.json` (see above).\n\n**NOTE:** In order to run unit test suites, you may need to manually switch to the correct virtual environment each time you wish to\nrun a different set of tests. To do this:\n\n- Show and Run Commands (Ctrl-Shift-P on Windows)\n    - Python: Create Environment\n    - Venv\n    - Select the `.venv` named for the test suite you wish to run, e.g. `backend`\n    - Use Existing\n- VSCode should now display a toast saying that the following environment is selected:\n    - (e.g.) `/mnt/d/Source/immunisation-fhir-api/backend/.venv/bin/python`\n\n### IntelliJ\n\n- Open the root repo directory in IntelliJ.\n- In Project Structure add an existing virtualenv SDK for `.direnv/python-x.x.x/bin/python`.\n- Set the project SDK and the default root module SDK to the one created above.\n    - Add `tests` as sources.\n    - Add `.direnv`, `infrastructure/instance/.terraform`, and `infrastructure/instance/build`\n      as exclusions if they're not already.\n- Add another existing virtualenv SDK for `backend/.direnv/python-x.x.x/bin/python`.\n- Import a module pointed at the `backend` directory, set the SDK created above.\n    - Add the `src` and `tests` directories as sources.\n    - Add `.direnv` as an exclusion if it's not already.\n\n## Verified commits\n\nPlease note that this project requires that all commits are verified using a GPG key.\nTo set up a GPG key please follow the instructions specified here:\nhttps://docs.github.com/en/authentication/managing-commit-signature-verification\n\n## AWS configuration: getting credentials for AWS federated user accounts\n\nIf you need to run commands that interact with AWS resources e.g. running a terraform plan against a dev environment locally\nthen you will need to configure AWS credentials.\n\nOnce you have been granted access, the `Access Keys` section within the AWS Access Portal will present you with several\noptions. It is _recommended_ to use `Option 2: Add a profile to your AWS credentials file`.\n\nThis is because the way that Python unittests using `moto` have been implemented is brittle and cannot handle other methods\nsuch as IAM Identity Centre SSO. In future, we should consider following [moto recommendations](https://docs.getmoto.org/en/latest/docs/getting_started.html#how-do-i-avoid-tests-from-mutating-my-real-infrastructure)\nto ensure our tests are authentication type agnostic and are fully robust.\n\nIf you _are_ using another option, such as SSO, and want to run unit tests then you will need to:\n\n- Add dummy values for `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` to the .env file in the Lambda dir you are testing.\n- Ensure those values are set before running the test, i.e. using the standard setup with direnv and a .envrc file\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fimmunisation-fhir-api","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnhsdigital%2Fimmunisation-fhir-api","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fimmunisation-fhir-api/lists"}