{"id":16078236,"url":"https://github.com/defra/find-ffa-data-ingester","last_synced_at":"2026-04-29T10:33:43.498Z","repository":{"id":257382682,"uuid":"858103257","full_name":"DEFRA/find-ffa-data-ingester","owner":"DEFRA","description":"Git repository for service find-ffa-data-ingester","archived":false,"fork":false,"pushed_at":"2024-11-20T10:52:12.000Z","size":557,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-04-05T10:43:01.061Z","etag":null,"topics":["backend","cdp","node","service"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DEFRA.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-16T10:09:23.000Z","updated_at":"2024-11-20T10:50:41.000Z","dependencies_parsed_at":"2024-10-31T02:40:54.182Z","dependency_job_id":"fe67f2be-98a7-4eeb-882f-a0a2dce237d0","html_url":"https://github.com/DEFRA/find-ffa-data-ingester","commit_stats":null,"previous_names":["defra/find-ffa-data-ingester"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffind-ffa-data-ingester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffind-ffa-data-ingester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffind-ffa-data-ingester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DEFRA%2Ffind-ffa-data-ingester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DEFRA","download_url":"https://codeload.github.com/DEFRA/find-ffa-data-ingester/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247325648,"owners_count":20920713,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["backend","cdp","node","service"],"created_at":"2024-10-09T10:09:16.613Z","updated_at":"2026-04-29T10:33:43.358Z","avatar_url":"https://github.com/DEFRA.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# find-ffa-data-ingester\n\nPopulates the FFA Vector store. Based on the Core delivery platform Node.js Backend Template.\n\n- [find-ffa-data-ingester](#find-ffa-data-ingester)\n  - [Requirements](#requirements)\n    - [Node.js](#nodejs)\n  - [Local development](#local-development)\n    - [Setup](#setup)\n    - [Development](#development)\n    - [Testing](#testing)\n    - [Production](#production)\n    - [Npm scripts](#npm-scripts)\n    - [Update dependencies](#update-dependencies)\n    - [Formatting](#formatting)\n      - [Windows prettier issue](#windows-prettier-issue)\n  - [Development helpers](#development-helpers)\n    - [Secrets \\\u0026 Environment Variables](#secrets--environment-variables)\n    - [MongoDB Locks](#mongodb-locks)\n  - [Docker](#docker)\n    - [Development image](#development-image)\n    - [Production image](#production-image)\n    - [Docker Compose](#docker-compose)\n    - [Dependabot](#dependabot)\n    - [SonarCloud](#sonarcloud)\n  - [Licence](#licence)\n    - [About the licence](#about-the-licence)\n\n## Requirements\n\n### Node.js\n\nTo use the correct version of Node.js for this application, via Node Version Manager [nvm](https://github.com/creationix/nvm):\n\n```bash\ncd find-ffa-data-ingester\nnvm use\n```\n\n## Local development\n\n### Setup\n\nInstall application dependencies:\n\n```bash\nnpm install\n```\n\nThen setup [Secrets \\\u0026 Environment Variables](#secrets--environment-variables)\n\n### Development\n\nOnce setup (above), get the dependencies up and running using [Docker Compose - see below](#docker-compose).\n\nThen run the application in `development` mode:\n\n```bash\nnpm run dev\n```\n\n### Testing\n\nTo test the application run:\n\n```bash\nnpm run test\n```\n\n### Production\n\nTo mimic the application running in `production` mode locally run:\n\n```bash\nnpm start\n```\n\n### Npm scripts\n\nAll available Npm scripts can be seen in [package.json](./package.json)\nTo view them in your command line run:\n\n```bash\nnpm run\n```\n\n### Update dependencies\n\nTo update dependencies use [npm-check-updates](https://github.com/raineorshine/npm-check-updates):\n\n\u003e The following script is a good start. Check out all the options on\n\u003e the [npm-check-updates](https://github.com/raineorshine/npm-check-updates)\n\n```bash\nncu --interactive --format group\n```\n\n### Formatting\n\n#### Windows prettier issue\n\nIf you are having issues with formatting of line breaks on Windows update your global git config by running:\n\n```bash\ngit config --global core.autocrlf false\n```\n\n## Development helpers\n\n### Secrets \u0026 Environment Variables\n\nCreate a `.env` file by making a copy of the `.env.example` and adding the missing secrets.\n\nDo not add secrets to `.env.example` as this is under version control!\n\n### MongoDB Locks\n\nIf you require a write lock for Mongo you can acquire it via `server.locker` or `request.locker`:\n\n```javascript\nasync function doStuff(server) {\n  const lock = await server.locker.lock('unique-resource-name')\n\n  if (!lock) {\n    // Lock unavailable\n    return\n  }\n\n  try {\n    // do stuff\n  } finally {\n    await lock.free()\n  }\n}\n```\n\nKeep it small and atomic.\n\nYou may use **using** for the lock resource management.\nNote test coverage reports do not like that syntax.\n\n```javascript\nasync function doStuff(server) {\n  await using lock = await server.locker.lock('unique-resource-name')\n\n  if (!lock) {\n    // Lock unavailable\n    return\n  }\n\n  // do stuff\n\n  // lock automatically released\n}\n```\n\nHelper methods are also available in `/src/helpers/mongo-lock.js`.\n\n## Docker\n\n### Development image\n\nBuild:\n\n```bash\ndocker build --target development --no-cache --tag find-ffa-data-ingester:development .\n```\n\nRun:\n\n```bash\ndocker run -e PORT=3001 -p 3001:3001 find-ffa-data-ingester:development\n```\n\n### Production image\n\nBuild:\n\n```bash\ndocker build --no-cache --tag find-ffa-data-ingester .\n```\n\nRun:\n\n```bash\ndocker run -e PORT=3001 -p 3001:3001 find-ffa-data-ingester\n```\n\n### Docker Compose\n\nA local environment with:\n\n- Localstack for AWS services (S3, SQS)\n- Redis\n- MongoDB\n\n```bash\ndocker compose up --build -d\n```\n\n### Dependabot\n\nWe have added an example dependabot configuration file to the repository. You can enable it by renaming\nthe [.github/example.dependabot.yml](.github/example.dependabot.yml) to `.github/dependabot.yml`\n\n### SonarCloud\n\nInstructions for setting up SonarCloud can be found in [sonar-project.properties](./sonar-project.properties)\n\n## Licence\n\nTHIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:\n\n\u003chttp://www.nationalarchives.gov.uk/doc/open-government-licence/version/3\u003e\n\nThe following attribution statement MUST be cited in your products and applications when using this information.\n\n\u003e Contains public sector information licensed under the Open Government license v3\n\n### About the licence\n\nThe Open Government Licence (OGL) was developed by the Controller of Her Majesty's Stationery Office (HMSO) to enable\ninformation providers in the public sector to license the use and re-use of their information under a common open\nlicence.\n\nIt is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffind-ffa-data-ingester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdefra%2Ffind-ffa-data-ingester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdefra%2Ffind-ffa-data-ingester/lists"}