{"id":25856609,"url":"https://github.com/defra/fcp-sfd-comms","last_synced_at":"2026-05-09T22:01:47.864Z","repository":{"id":278547409,"uuid":"935984125","full_name":"DEFRA/fcp-sfd-comms","owner":"DEFRA","description":"Git repository for service fcp-sfd-comms","archived":false,"fork":false,"pushed_at":"2025-02-20T10:43:58.000Z","size":0,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-20T11:33:03.557Z","etag":null,"topics":["backend","cdp","node","service"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DEFRA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-02-20T10:43:42.000Z","updated_at":"2025-02-20T10:44:01.000Z","dependencies_parsed_at":"2025-02-20T11:44:53.702Z","dependency_job_id":null,"html_url":"https://github.com/DEFRA/fcp-sfd-comms","commit_stats":null,"previous_names":["defra/fcp-sfd-comms"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffcp-sfd-comms","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffcp-sfd-comms/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffcp-sfd-comms/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffcp-sfd-comms/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DEFRA","download_url":"https://codeload.github.com/DEFRA/fcp-sfd-comms/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241404565,"owners_count":19957676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","cdp","node","service"],"created_at":"2025-03-01T18:19:12.191Z","updated_at":"2026-05-09T22:01:47.858Z","avatar_url":"https://github.com/DEFRA.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fcp-sfd-comms\n![Publish](https://github.com/defra/fcp-sfd-comms/actions/workflows/publish.yml/badge.svg)\n[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=DEFRA_fcp-sfd-comms\u0026metric=alert_status)](https://sonarcloud.io/summary/new_code?id=DEFRA_fcp-sfd-comms)\n[![Coverage](https://sonarcloud.io/api/project_badges/measure?project=DEFRA_fcp-sfd-comms\u0026metric=coverage)](https://sonarcloud.io/summary/new_code?id=DEFRA_fcp-sfd-comms)\n[![Maintainability Rating](https://sonarcloud.io/api/project_badges/measure?project=DEFRA_fcp-sfd-comms\u0026metric=sqale_rating)](https://sonarcloud.io/summary/new_code?id=DEFRA_fcp-sfd-comms)\n\nCommunications service for the Single Front Door.\n\nThis service is part of the [Single Front Door (SFD) service](https://github.com/DEFRA/fcp-sfd-core).\n\n## Architecture overview\n\n## Event processing pipeline\n\n\u003e **API Specification**\n\u003e Complete AsyncAPI specification is available at: [`docs/asyncapi0-v1.0.yaml`](/docs/asyncapi-v1.0.yaml).\n\u003e Defines all supported inbound events and schemas with examples provided.\n\n### Processing flow inbound messages\n\n```mermaid\nsequenceDiagram\n    participant CONSUMER as Consumer\n    participant SQS as SFD request queue\n    participant SFD as Single Front Door (fcp-sfd-comms)\n    participant MONGO as MongoDB\n    participant SNS as SFD events topic\n    participant NOTIFY as GOV.UK Notify API\n    participant FDM as Farming Data Model (fcp-fdm)\n    participant RECIPIENT as Recipient inbox\n\n    rect\n        note over CONSUMER,SNS: Inbound message\n    end\n\n    CONSUMER-\u003e\u003eSQS: 1. Send message\n    SQS-\u003e\u003eSFD: 2. Parse message\n    SFD-\u003e\u003eMONGO: 3. Validate and store message\n    SFD-\u003e\u003eSNS: 4. Build and publish message\n\n    rect\n        note over SNS,RECIPIENT: Outbound message\n    end\n\n    SFD-\u003e\u003eNOTIFY: 5. Send request to Notify\n    FDM--\u003e\u003eSNS: 6. Listens for message\n    NOTIFY-\u003e\u003eRECIPIENT: 7. Deliver message\n\n    SFD-\u003e\u003eNOTIFY: 8. Retrieve status update\n    NOTIFY--\u003e\u003eSFD: 9. Return status update\n    SFD-\u003e\u003eSNS: 10. Build and publish message with status update\n    FDM--\u003e\u003eSNS: 11. Listens for message with status update\n    SFD-\u003e\u003eMONGO: 12. Store message with status update\n\n    rect\n        note over SFD,RECIPIENT: Retry logic\n    end\n\n    SFD--\u003e\u003eSFD: i. Handle retries\n    SFD-\u003e\u003eSNS: ii. Re-build and publish message on retry\n    FDM--\u003e\u003eSNS: iii. Listen for message\n    SFD-\u003e\u003eMONGO: iv. Store message\n    SFD-\u003e\u003eNOTIFY: v. Send retry request to Notify\n    NOTIFY-\u003e\u003eRECIPIENT: vi. Deliver message on retry\n```\n\n### Message processing stages\n\n1. Consumer sends request onto the Single Front Door (SFD) queue: `fcp_sfd_comms_request`.\n2. Message is consumed and parsed by the SFD comms service: `fcp-sfd-comms`.\n3. Comms service will validate the request against the message schema, check the request being made is idempotent to avoid duplicate requests, and then store the valid message in the SFD's comms database: `fcp_sfd_comms`.\n4. Comms builds the message to then be published onto SFD's topic: `fcp_sfd_comm_events`.\n5. All topic subscriptions will receive the message. The Farming Data Model (FDM) is currently the only subscriber to SFD's topic.\n6. Comms service sends a request to the GOV.UK Notify API alongside the built message which includes the Notify template ID.\n7. Message is delivered to the recipient.\n\nExtended details on the cron job for status update retrieval and retry logic is provided below.\n\n### Processing flow for status update retrieval\n\n```mermaid\nsequenceDiagram\n    participant SFD as Single Front Door (SFD)\n    participant NOTIFY as GOV.UK Notify API\n    participant MONGO as MongoDB\n    participant SNS as SFD events topic\n    participant FDM as Farming Data Model (FDM)\n\n    loop Cron job every 30 seconds\n        SFD-\u003e\u003eNOTIFY: 1. Retrieve status update\n        NOTIFY--\u003e\u003eSFD: 2. Return status update\n        SFD-\u003e\u003eSNS: 3. Build and publish message with status update\n        SFD-\u003e\u003eMONGO: 4. Store status update\n        FDM--\u003e\u003eSNS: 5. Listen for message with status update\n\n        alt Message status failed, proceed to retry\n            SFD--\u003e\u003eSFD: i. Handle retries\n            SFD-\u003e\u003eNOTIFY: ii. Send retry request to Notify\n            SFD-\u003e\u003eSNS: iii. Re-build and publish on retry\n            SFD-\u003e\u003eMONGO: iv. Store message\n            FDM--\u003e\u003eSNS: v. Listen for message on retry\n        end\n    end\n```\n\n### Status retrieval and retry logic stages\n\n1. Cron job is triggered to run every 30 seconds. First stage is sending a request to the GOV.UK Notify API for a status update.\n2. Notify API returns status update to SFD's comms service.\n3. Comms service will build and publish the message with the status update onto the SFD topic.\n4. Updated message is stored in SFD's comms database.\n5. All topic subscribers will listen and consume the updated message.\n\nThe `fcp-sfd-comms` service is also configured to handle retries (i) on any messages that fail to be delivered to the recipient. Requests are sent to the Notify API to retry sending the message (ii). In parallel to this, `fcp-sfd-comms` will re-build the message and publish it onto the SFD topic (iii) and store the message in the Mongo database (iv). All topic subscriptions (in this case FDM) will consume the message published onto SFD's topic (v).\n\n## Prerequisites\n- Docker\n- Docker Compose\n- Node.js (v22 LTS)\n\n### SonarQube Cloud token\n\nOne of the npm scripts configured for this service enables code scanning by SonarQube Cloud. This will look for any issues and can be ran optionally before committing if the developer wishes to resolve issues during local development. This script helps ensure fewer issues are pushed to GitHub leading to earlier resolution of existing vulnerabilities. In order for this script to run successfully during local development you will need to generate your own personal `SONAR_TOKEN` and add it to your `.env`:\n\n- Log into [SonarQube Cloud](https://sonarcloud.io/login).\n- Navigate to your `My Account` settings.\n- On the left-hand sidebar navigate to the `Security` tab.\n- Under `Generate Tokens` enter a name for your token and click `Generate Token`.\n- Copy the token and add it to your `.env`, referring to it as [`SONAR_TOKEN`](.env.example).\n\n## Pre-commit Hooks\n\nFor local development, this repository includes [`pre-commit` hooks](https://pre-commit.com/). These hooks allow for early identification of issues and vulnerabilities so that the developer can resolve any issues before pushing up to the public repository on GitHub. The hooks include:\n\n- [`detect-secrets`](https://github.com/Yelp/detect-secrets): for detecting and preventing secrets in the codebase being pushed to public/open-source repositories.\n- `eslint-fix`: a custom hook for running the linter, ESLint + [neostandard](https://www.npmjs.com/package/neostandard?activeTab=readme), to ensure consistent code formatting and styling and additionally uses the `--fix` option to automatically fix any identified issues where possible to reduce the need for manual correction.\n\nTo see the full output of the above hooks it is recommended to commit via the command line as using the source control panel does not provide the same feedback and loses sight of the `pre-commit` logs. All `pre-commit` hooks are listed in the [`.pre-commit-config.yaml`](.pre-commit-config.yaml) configuration file.\n\nFor these hooks to successfully apply during local development ensure  Python and its package manager, `pip3`, are installed on your machine. Installation of `pre-commit` can then be completed via `pip3`:\n\n```\npip3 install pre-commit\n```\n\n## Environment variables\n| Name | Default value | Required | Description |\n|-|-|-|-|\n| AWS_REGION | eu-west-2 | No | AWS region to access resources in. |\n| AWS_DEFAULT_REGION | eu-west-2 | No | Default AWS region to access resources in. |\n| AWS_ACCESS_KEY_ID | test | No | AWS Access Key ID. |\n| AWS_SECRET_ACCESS_KEY | test | No | AWS Secret Access Key. |\n| NOTIFY_API_KEY | n/a - sensitive | Yes | Notify API key to perform API requests to GOV.UK Notify. Generate a test API key for local development and testing on the [GOV.UK Notify portal](https://www.notifications.service.gov.uk/). |\n| MONGO_URI | mongodb://mongo:27017/ | No | MongoDB connection string. |\n| SQS_ENDPOINT | http://localstack:4566 | No | SQS endpoint. |\n| SNS_ENDPOINT | http://localstack:4566 | No | SNS endpoint. |\n| COMMS_REQUEST_QUEUE_URL | http://sqs.eu-west-2.127.0.0.1:4566/000000000000/fcp_sfd_comms_request | No | SQS queue URL to send comms requests. |\n| COMMS_REQUEST_DEAD_LETTER_QUEUE_URL | http://sqs.eu-west-2.127.0.0.1:4566/000000000000/fcp_sfd_comms_request-deadletter | No | Comms Request SQS dead letter queue. |\n| COMM_EVENTS_TOPIC_ARN | arn:aws:sns:eu-west-2:000000000000:fcp_sfd_comm_events | No | SNS topic ARN to publish comm events to. |\n| FDM_QUEUE_URL | http://sqs.eu-west-2.127.0.0.1:4566/000000000000/fcp_fdm_events | No | SQS queue that is subscribed to the events SNS topic. |\n\n## Running the application\n\nWe recommend using the [fcp-sfd-core](https://github.com/DEFRA/fcp-sfd-core) repository for local development. You can however run this service independently by following the instructions below using either Docker Compose or the provided [npm scripts](./package.json). Alternatively, for VS Code users, a set of [VS Code tasks](.vscode/tasks.json) are available to use and can be access via the command palette: \n\n- `Ctrl` + `shift` + `P` on Windows or `Cmd` + `shift` + `P` on Mac.\n- Select `Tasks: Run Task`.\n- Choose from the available tasks listed.\n\n\u003e ℹ️️ The aws resources created locally are meant to be shared with [fcp-sfd-comms-publisher-stub](https://github.com/DEFRA/fcp-sfd-comms-publisher-stub) when running alongside.\n\u003e fcp-sfd-comms can be run independently of fcp-sfd-comms-publisher-stub.\n\n### Build container image\n\nContainer images are built using Docker Compose.\n\n```\ndocker compose build\n```\n\nAlternatively, an npm script is available:\n\n```\nnpm run docker:build\n```\n\n### Start\n\nUse Docker Compose to start running the service locally.\n\n```\ndocker compose up\n```\n\nAlternatively, an npm script is available:\n\n```\nnpm run docker:dev\n```\n\n### Test structure\n\nThe tests have been structured into sub-folders of `./test` as per the\n[Microservice test approach and repository structure](https://eaflood.atlassian.net/wiki/spaces/FPS/pages/1845396477/Microservice+test+approach+and+repository+structure). \n\n### Running tests\n\nA convenience npm script is provided to run automated tests in a containerised\nenvironment. This will rebuild images before running tests via Docker Compose,\nusing a combination of the `compose.yaml` and `compose.test.yaml` files.\n\n```\nnpm run docker:test\n```\n\nTests can also be started in watch mode to support Test Driven Development (TDD):\n\n```\nnpm run docker:test:watch\n```\n\nAs mentioned previously, Docker Compose can be used directly for starting tests:\n\n```\ndocker compose -f compose.yaml -f compose.test.yaml run --rm \"fcp-sfd-comms\"\n```\n\n\n## Licence\n\nTHIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:\n\n\u003chttp://www.nationalarchives.gov.uk/doc/open-government-licence/version/3\u003e\n\nThe following attribution statement MUST be cited in your products and applications when using this information.\n\n\u003e Contains public sector information licensed under the Open Government license v3\n\n### About the licence\n\nThe Open Government Licence (OGL) was developed by the Controller of His Majesty's Stationery Office (HMSO) to enable information providers in the public sector to license the use and re-use of their information under a common open licence.\n\nIt is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffcp-sfd-comms","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdefra%2Ffcp-sfd-comms","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffcp-sfd-comms/lists"}