{"id":23421197,"url":"https://github.com/nhsdigital/cvd-prevent-tool","last_synced_at":"2026-03-11T10:06:31.639Z","repository":{"id":192857872,"uuid":"687540596","full_name":"NHSDigital/cvd-prevent-tool","owner":"NHSDigital","description":"This repo includes the pipeline used to link and curate CVD PREVENT audit data to HES and death registration data into two tables. These are subsequently sent to OHID for analysis and publication.","archived":false,"fork":false,"pushed_at":"2024-09-04T12:09:34.000Z","size":2976,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-15T12:44:47.898Z","etag":null,"topics":["cvd","death-registration","healthcare","hes","nhs","nhs-england","nhse","reproducible-analytical-pipeline"],"latest_commit_sha":null,"homepage":"https://www.cvdprevent.nhs.uk/home","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NHSDigital.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-05T15:06:02.000Z","updated_at":"2024-12-12T14:42:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"3366f54a-5af4-4bd5-af5a-aecd932c2677","html_url":"https://github.com/NHSDigital/cvd-prevent-tool","commit_stats":null,"previous_names":["nhsdigital/cvd-prevent-tool"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/NHSDigital/cvd-prevent-tool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fcvd-prevent-tool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fcvd-prevent-tool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fcvd-prevent-tool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fcvd-prevent-tool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NHSDigital","download_url":"https://codeload.github.com/NHSDigital/cvd-prevent-tool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NHSDigital%2Fcvd-prevent-tool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30377838,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-11T06:09:32.197Z","status":"ssl_error","status_checked_at":"2026-03-11T06:09:17.086Z","response_time":84,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cvd","death-registration","healthcare","hes","nhs","nhs-england","nhse","reproducible-analytical-pipeline"],"created_at":"2024-12-23T02:14:23.738Z","updated_at":"2026-03-11T10:06:31.606Z","avatar_url":"https://github.com/NHSDigital.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DS\\_234: README\n\n**CVD Prevent Tool curated data pipeline**\n\n**Repository owner: [NHS England Analytical Services](https://github.com/NHSDigital/data-analytics-services)**\n\n***Email: [datascience@nhs.net](mailto:datascience@nhs.net)***\n\n***To contact us, raise an issue on Github or via email and we will respond promptly.***\n\n***Warning - this repository is a snapshot of a repository internal to NHS England.\nThis means that some links may not work for external readers.***\n\n- [Key features](#key-features)\n- [What does the pipeline do](#what-does-the-pipeline-do)\n- [Quickstart](#quickstart)\n  - [Running the pipeline](#running-the-pipeline)\n- [Documentation](#documentation)\n  - [Further documentation](#further-documentation)\n- [Licence](#licence)\n\nThis repository includes a suite of spark notebooks used to build a new data pipeline. These are used to build a new data asset to link and curate [CVDPREVENT audit data](https://www.cvdprevent.nhs.uk/home) to existing administrative data tables.\n\nThis codebase can only be run on NHS England's Data Access Environment Apache Spark *V3.2.1*. This is being shared for transparency and feedback on the algorithms used.\n\nNo sensitive data is stored within this repository.\n\n## Key features\n\n- The pipeline is structured using an object-oriented approach.\n- The pipeline is designed to be configured, using *params* notebooks, without altering the codebase\n- Outputs can be restricted for particular cohort populations or to include a subset of data sources.\n- Potential to include bespoke outcomes and patient characteristics in the output tables\n\n## What does the pipeline do\n\nIt takes information from a range of data sources, and summarises them in a number of standardised tables. The pipeline produces the following outputs:\n\n  - Events table (row per event from each data source used)\n  - Patient table (row per patient – patient only recorded if they satisfy inclusion criteria for either Cohort 1 or 2)\n  - Report table (output of results check, error catching)\n\n\n\n#  Quick Start Guide\n\nThe Prevent Tool Pipeline is run from the run_pipeline notebook.\n\nThis notebook will run the full pipeline run and uses several haardcoded parameters to determine how the pipeline is run:\n\n**PARAMS\\_PATH:**\n Path to the parameters notebook that controls the pipeline. Default is default. A custom path should only be used when using a non-standard parameters file.\n\n**VERSION:**\n Git commit hash from the current master branch in gitlab. Can also be set to dev\\_XX where XX are the initials of the user running the pipeline - used when testing pipeline code.\n\n**RUN\\_LOGGER:**\n Boolean (True or False) of if to run the logger stage of the pipeline. If True the stage produces metadata around the pipeline's written assets, this information is written into a seperate logger_table asset.\n\n**RUN\\_ARCHIVE:**\n Boolean (True or False) of if to run the archive stage of the pipeline. If True the stage  copies current pipeline assets wiith date and git hash before overwriting the assets with the new versions.\n\n**DEV\\_MODE:**\n Boolean (True or False) of if to run the pipeline in development mode. If True the pipeline assets are written with the prefix \\_dev.\n\nThe pipeline run function run\\_pipeline() outputs a verbose progress log of the running stages and times of the pipeline.\n\nOnce completed, assets will be available in the prevent\\_tool\\_collab database.\n\n\n\n# Configuration\n\nThe pipeline functionality and running can be controlled using the pipeline parameters (found in the params folder). Below is a brief summary of the different parameter notebooks and their purpose.\n\n**params**\n\nThe main notebook for creating the params object. This notebook checks for the parameters path (default is default) and loads the specified params\\_util notebook.\n\n**params\\_util**\n\nThis notebook contains the main parameter definitions and the creation of the params dataclass.\n Input and output data fields (columns) are specified here, alongside any intermediate fields used as part of the pipeline processing.\n This notebook loads the params\\_diagnostic\\_codes notebook to load the relevant SNOMED and ICD10 codes that form part of the inclusion criteria.\n\n**params\\_diagnostic\\_codes**\n\nThis notebook is used to specify any clinical coding variables (ICD-10, SNOMED) that are used to create the pipeline parameters.\n\n**params\\_pipeline\\_assets**\n\nThis notebook is used to specify the input (pipeline parameters) and output (table names) used in the pipeline stages.\n\n**params\\_table\\_schemas**\n\nThis notebook is used to specify the expected final schemas for the events and patient table assets.\n\n## Documentation\n\nThe homepage of the pipeline's documentation is [here](./documentation/pipeline_README.md). \n\n### Further documentation\n\n[Configuring the pipeline](./documentation/pipeline_README.md)\n\n[Output data specification](./documentation/CVD_prevent_tool_product-spec_v1.4_extended.2.xlsx)\n\n\n## Licence\n\nUnless stated otherwise, the codebase is released under the [MIT Licence](./LICENSE). This covers both the codebase and any sample code in the documentation.\n\nDocumentation is © Crown copyright and available under the terms of the [Open Government 3.0 licence](https://www.nationalarchives.gov.uk/doc/open-government-licence/version/3/).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fcvd-prevent-tool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnhsdigital%2Fcvd-prevent-tool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnhsdigital%2Fcvd-prevent-tool/lists"}