{"id":26712314,"url":"https://github.com/defra/fg-cw-backend","last_synced_at":"2025-08-13T21:41:53.593Z","repository":{"id":284540482,"uuid":"955270568","full_name":"DEFRA/fg-cw-backend","owner":"DEFRA","description":"Git repository for service fg-cw-backend","archived":false,"fork":false,"pushed_at":"2025-08-08T10:05:01.000Z","size":1143,"stargazers_count":0,"open_issues_count":10,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-08T11:27:21.094Z","etag":null,"topics":["backend","cdp","node","service"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DEFRA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-26T11:34:17.000Z","updated_at":"2025-08-08T10:03:51.000Z","dependencies_parsed_at":"2025-04-10T15:40:03.481Z","dependency_job_id":"d01070c8-c681-41e6-b852-deaa5bdcb080","html_url":"https://github.com/DEFRA/fg-cw-backend","commit_stats":null,"previous_names":["defra/fg-cw-backend"],"tags_count":90,"template":false,"template_full_name":null,"purl":"pkg:github/DEFRA/fg-cw-backend","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffg-cw-backend","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffg-cw-backend/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffg-cw-backend/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffg-cw-backend/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DEFRA","download_url":"https://codeload.github.com/DEFRA/fg-cw-backend/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffg-cw-backend/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":270322775,"owners_count":24564643,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-13T02:00:09.904Z","response_time":66,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","cdp","node","service"],"created_at":"2025-03-27T11:23:46.853Z","updated_at":"2025-08-13T21:41:53.550Z","avatar_url":"https://github.com/DEFRA.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fg-cw-backend\n\n## AWS\n\n#### Get a topics attributes (fg-gas-backend DEV)\n\nNote that the SQS url http://sqs.eu-west-2.127.0.0.1:4566 will also work in the terminal\n\n```bash\naws sns get-topic-attributes --topic-arn arn:aws:sns:eu-west-2:332499610595:grant_application_created\n```\n\n#### Get a queue attributes\n\n```\naws sqs get-queue-attributes \\\n  --queue-url https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case --attribute-names All\n```\n\n```\naws sqs get-queue-attributes \\\n  --queue-url https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case-deadletter --attribute-names All\n```\n\n#### Send message to the grant_application_created topic\n\n```\naws sns publish \\\n  --topic-arn \"arn:aws:sns:eu-west-2:332499610595:grant_application_created\" \\\n  --message '{\"clientRef\": \"APPLICATION-REF-1\", \"code\": \"frps-private-beta\", \"createdAt\": \"2025-03-27T10:34:52.000Z\", \"submittedAt\": \"2025-03-28T11:30:52.000Z\", \"identifiers\": { \"sbi\": \"SBI001\", \"frn\": \"FIRM0001\", \"crn\": \"CUST0001\", \"defraId\": \"DEFRA0001\" }, \"answers\": { \"scheme\": \"SFI\", \"year\": 2025, \"hasCheckedLandIsUpToDate\": true, \"actionApplications\": [ { \"parcelId\": \"9238\", \"sheetId\": \"SX0679\", \"code\": \"CSAM1\", \"appliedFor\": {\"unit\": \"ha\",\"quantity\": 20.23 }}]}}'\n```\n\n#### Check the message has arrived in the queue\n\n```\naws sqs receive-message \\\n--queue-url \"https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case\"\n```\n\n```\naws sqs receive-message \\\n--queue-url \"https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case-deadletter\"\n```\n\n#### Delete a message from the queue\n\n```\naws sqs delete-message \\\n--queue-url https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case --receipt-handle \u003creceipt-handle\u003e\n```\n\n#### Purge the queue\n\n```\naws sqs purge-queue \\\n--queue-url https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case\n```\n\n```\naws sqs purge-queue \\\n--queue-url https://sqs.eu-west-2.amazonaws.com/332499610595/create_new_case-deadletter\n```\n\n#### Test the dead letter queue\n\nSend a message in and try to receive the message four times like so\n\n```\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\n```\n\n## Docker\n\nLaunch CW and dependencies via Docker Compose:\n\n```\ndocker compose up --watch --build\n```\n\nCheck the container status in our system and see the container id's\n\n```\ndocker ps -a\n```\n\nRun an interactive shell on a container\n\n```\ndocker exec -it \u003ccontainer id\u003e sh\n```\n\n## Local stack\n\n### Useful commands\n\nDocker compose uses localstack to replicate the aws environment.\nHere are some useful commands for interacting with the localstack aws.\n\n#### List the topics\n\n`awslocal sns list-topics`\n\n#### Get a topics attributes\n\n```bash\nawslocal sns get-topic-attributes --topic-arn arn:aws:sns:eu-west-2:000000000000:grant_application_created\n```\n\n#### Get a queue attributes\n\n```\nawslocal sqs get-queue-attributes \\\n  --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case --attribute-names All\n```\n\n```\nawslocal sqs get-queue-attributes \\\n  --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case-deadletter --attribute-names All\n```\n\n#### Send message to the grant_application_created topic\n\n```\nawslocal sns publish \\\n  --topic-arn \"arn:aws:sns:eu-west-2:000000000000:grant_application_created\" \\\n  --message '{\"clientRef\": \"APPLICATION-REF-2\",\n  \"code\": \"frps-private-beta\",\n  \"createdAt\": \"2025-03-27T10:34:52.000Z\",\n  \"submittedAt\": \"2025-03-28T11:30:52.000Z\",\n  \"identifiers\": {\n    \"sbi\": \"SBI001\",\n    \"frn\": \"FIRM0001\",\n    \"crn\": \"CUST0001\",\n    \"defraId\": \"DEFRA0001\"\n  },\n  \"answers\": {\n    \"scheme\": \"SFI\",\n    \"year\": 2025,\n    \"hasCheckedLandIsUpToDate\": true,\n    \"actionApplications\": [\n      {\n        \"parcelId\": \"9238\",\n        \"sheetId\": \"SX0679\",\n        \"code\": \"CSAM1\",\n        \"appliedFor\": {\n          \"unit\": \"ha\",\n          \"quantity\": 20.23\n        }\n      }\n    ]\n  }\n}'\n```\n\n#### Check the message has arrived in the queue\n\n```\nawslocal sqs receive-message \\\n--queue-url \"http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\"\n```\n\n```\nawslocal sqs receive-message \\\n--queue-url \"http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case-deadletter\"\n```\n\n#### Delete a message from the queue\n\n```\nawslocal sqs delete-message \\\n--queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case --receipt-handle \u003creceipt-handle\u003e\n```\n\n#### Purge the queue\n\n```\nawslocal sqs purge-queue \\\n--queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\n```\n\n#### Test the dead letter queue\n\nSend a message in and try to receive the message four times like so\n\n```\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\nawslocal sqs receive-message --visibility-timeout 0 --queue-url http://sqs.eu-west-2.127.0.0.1:4566/000000000000/create_new_case\n```\n\n#### SQS Retry Mechanism\n\nThe application implements an automatic retry mechanism for failed SQS message processing:\n\n1. When a message fails to process, it will be retried based on the `maxRetries` configuration (default: 3)\n2. Each retry attempt uses exponential backoff (30s, 60s, 120s, etc.)\n3. After all retry attempts are exhausted, the message is moved to the Dead Letter Queue (DLQ)\n4. The retry count is tracked using SQS's built-in `ApproximateReceiveCount` attribute\n\nTo configure the maximum number of retries, set the `SQS_MAX_RETRIES` environment variable or update the default in `src/config.js`.\n\n#### Move the DLQ messages back into the recovery queue\n\n```\nawslocal sqs start-message-move-task \\\n  --source-arn arn:aws:sqs:eu-west-2:000000000000:create_new_case-deadletter \\\n  --destination-arn arn:aws:sqs:eu-west-2:000000000000:create_new_case-recovery\n```\n\nCore delivery platform Node.js Backend Template.\n\n- [Requirements](#requirements)\n  - [Node.js](#nodejs)\n- [Local development](#local-development)\n  - [Setup](#setup)\n  - [Development](#development)\n  - [Testing](#testing)\n  - [Production](#production)\n  - [Npm scripts](#npm-scripts)\n  - [Update dependencies](#update-dependencies)\n  - [Formatting](#formatting)\n    - [Windows prettier issue](#windows-prettier-issue)\n- [API endpoints](#api-endpoints)\n- [Development helpers](#development-helpers)\n  - [MongoDB Locks](#mongodb-locks)\n  - [Proxy](#proxy)\n- [Docker](#docker)\n  - [Development image](#development-image)\n  - [Production image](#production-image)\n  - [Docker Compose](#docker-compose)\n  - [Dependabot](#dependabot)\n  - [SonarCloud](#sonarcloud)\n- [Licence](#licence)\n  - [About the licence](#about-the-licence)\n\n## Requirements\n\n### Node.js\n\nPlease install [Node.js](http://nodejs.org/) `\u003e= v22` and [npm](https://nodejs.org/) `\u003e= v11`. You will find it\neasier to use the Node Version Manager [nvm](https://github.com/creationix/nvm)\n\nTo use the correct version of Node.js for this application, via nvm:\n\n```bash\ncd fg-cw-backend\nnvm use\n```\n\n## Local development\n\n### Setup\n\nInstall application dependencies:\n\n```bash\nnpm install\n```\n\n### Development\n\nTo run the application in `development` mode run:\n\n```bash\ndocker compose up --watch --build\n```\n\n### Testing\n\nTo test the application run:\n\n```bash\nnpm run test\n```\n\nor (with coverage)\n\n```bash\nnpm run coverage\n```\n\nTo run integration tests:\n\n```bash\nnpm run test:integration\n```\n\n### Production\n\nTo mimic the application running in `production` mode locally run:\n\n```bash\nnpm start\n```\n\n### Npm scripts\n\nAll available Npm scripts can be seen in [package.json](./package.json).\nTo view them in your command line run:\n\n```bash\nnpm run\n```\n\n#### Publish SQS messages\n\nCreate a new case:\n\n```bash\nnpm run publish:case:new\n```\n\n### Update dependencies\n\nTo update dependencies use [npm-check-updates](https://github.com/raineorshine/npm-check-updates):\n\n\u003e The following script is a good start. Check out all the options on\n\u003e the [npm-check-updates](https://github.com/raineorshine/npm-check-updates)\n\n```bash\nncu --interactive --format group\n```\n\n### Formatting\n\n#### Windows prettier issue\n\nIf you are having issues with formatting of line breaks on Windows update your global git config by running:\n\n```bash\ngit config --global core.autocrlf false\n```\n\n## API endpoints\n\n| Endpoint             | Description                    |\n| :------------------- | :----------------------------- |\n| `GET: /health`       | Health                         |\n| `GET: /example    `  | Example API (remove as needed) |\n| `GET: /example/\u003cid\u003e` | Example API (remove as needed) |\n\n## Development helpers\n\n### MongoDB Locks\n\nIf you require a write lock for Mongo you can acquire it via `server.locker` or `request.locker`:\n\n```javascript\nasync function doStuff(server) {\n  const lock = await server.locker.lock(\"unique-resource-name\");\n\n  if (!lock) {\n    // Lock unavailable\n    return;\n  }\n\n  try {\n    // do stuff\n  } finally {\n    await lock.free();\n  }\n}\n```\n\nKeep it small and atomic.\n\nYou may use **using** for the lock resource management.\nNote test coverage reports do not like that syntax.\n\n```javascript\nasync function doStuff(server) {\n  await using lock = await server.locker.lock(\"unique-resource-name\");\n\n  if (!lock) {\n    // Lock unavailable\n    return;\n  }\n\n  // do stuff\n\n  // lock automatically released\n}\n```\n\nHelper methods are also available in `/src/helpers/mongo-lock.js`.\n\n### Proxy\n\nWe are using forward-proxy which is set up by default. To make use of this: `import { fetch } from 'undici'` then\nbecause of the `setGlobalDispatcher(new ProxyAgent(proxyUrl))` calls will use the ProxyAgent Dispatcher\n\nIf you are not using Wreck, Axios or Undici or a similar http that uses `Request`. Then you may have to provide the\nproxy dispatcher:\n\nTo add the dispatcher to your own client:\n\n```javascript\nimport { ProxyAgent } from \"undici\";\n\nreturn await fetch(url, {\n  dispatcher: new ProxyAgent({\n    uri: proxyUrl,\n    keepAliveTimeout: 10,\n    keepAliveMaxTimeout: 10,\n  }),\n});\n```\n\n## Docker\n\n### Development image\n\nBuild:\n\n```bash\ndocker build --target development --no-cache --tag fg-cw-backend:development .\n```\n\nRun:\n\n```bash\ndocker run -e PORT=3001 -p 3001:3001 fg-cw-backend:development\n```\n\n### Production image\n\nBuild:\n\n```bash\ndocker build --no-cache --tag fg-cw-backend .\n```\n\nRun:\n\n```bash\ndocker run -e PORT=3001 -p 3001:3001 fg-cw-backend\n```\n\n### Docker Compose\n\nA local environment with:\n\n- Localstack for AWS services (S3, SQS)\n- MongoDB\n- This service.\n- A commented out frontend example.\n\n```bash\ndocker compose up --build -d\n```\n\n### Dependabot\n\nWe have added an example dependabot configuration file to the repository. You can enable it by renaming\nthe [.github/example.dependabot.yml](.github/dependabot.yml) to `.github/dependabot.yml`\n\n### SonarCloud\n\nInstructions for setting up SonarCloud can be found in [sonar-project.properties](./sonar-project.properties)\n\n## Licence\n\nTHIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:\n\n\u003chttp://www.nationalarchives.gov.uk/doc/open-government-licence/version/3\u003e\n\nThe following attribution statement MUST be cited in your products and applications when using this information.\n\n\u003e Contains public sector information licensed under the Open Government license v3\n\n### About the licence\n\nThe Open Government Licence (OGL) was developed by the Controller of Her Majesty's Stationery Office (HMSO) to enable\ninformation providers in the public sector to license the use and re-use of their information under a common open\nlicence.\n\nIt is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffg-cw-backend","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdefra%2Ffg-cw-backend","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffg-cw-backend/lists"}