{"id":23067022,"url":"https://github.com/dvsa/ftts-payments-sap-sftp-push","last_synced_at":"2025-04-03T09:13:58.561Z","repository":{"id":44051601,"uuid":"480781180","full_name":"dvsa/ftts-payments-sap-sftp-push","owner":"dvsa","description":null,"archived":false,"fork":false,"pushed_at":"2023-01-01T16:19:09.000Z","size":127,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-02-08T22:28:35.400Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dvsa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-04-12T11:29:56.000Z","updated_at":"2024-01-03T09:22:10.000Z","dependencies_parsed_at":"2023-01-31T23:45:31.394Z","dependency_job_id":null,"html_url":"https://github.com/dvsa/ftts-payments-sap-sftp-push","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvsa%2Fftts-payments-sap-sftp-push","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvsa%2Fftts-payments-sap-sftp-push/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvsa%2Fftts-payments-sap-sftp-push/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dvsa%2Fftts-payments-sap-sftp-push/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dvsa","download_url":"https://codeload.github.com/dvsa/ftts-payments-sap-sftp-push/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246970323,"owners_count":20862509,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-12-16T05:15:56.599Z","updated_at":"2025-04-03T09:13:58.539Z","avatar_url":"https://github.com/dvsa.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# FTTS-PAYMENTS-SAP-SFTP-PUSH\n\n## Getting Started\n\nProject is based on Node.js and Azure Functions.\n\n### Dependencies\n\n- Node.js installed on local machine (v16.18.1) https://nodejs.org/en/\n- The following packages may need to be installed globally (`npm install -g`) to avoid errors:\n- azure-functions-core-tools@3.0.2245\n- azurite https://github.com/Azure/Azurite\n### Settings\n\nCreate local-settings.json file in the project root by running `npm run copy-config`\n\n## Local storage\n\nTo run SFTP Push app on local machine you can use storage emulator or point to the deployed sapfilestor.\n\nOn Windows it can be Azure Storage Emulator https://docs.microsoft.com/en-us/azure/storage/common/storage-use-emulator\n\nOn Mac OS or Linux you can use Azurite https://github.com/Azure/Azurite. Use the VSCode Azurite extension and launch 'Blob Service'.\nAlternatively use Docker:\n\n```bash\ndocker pull mcr.microsoft.com/azure-storage/azurite\n```\n\nand then\n\n```bash\ndocker run -p 10000:10000 -p 10001:10001 mcr.microsoft.com/azure-storage/azurite\n```\n\n## Testing on local environment\n\n### 1. Launch DNS and SFTP servers\n\nOutside application directory:\n\n```bash\nmkdir dnsmasq\ncd dnsmasq\ntouch dnsmasq.conf\ntouch docker-compose.yml\n```\n\nNext edit `dnsmasq.conf` and put\n\n```bash\n#log all dns queries\nlog-queries\n#dont use hosts nameservers\nno-resolv\n#use google as default nameservers\nserver=8.8.4.4\nserver=8.8.8.8\n#explicitly define host-ip mappings\naddress=/sftp_server/127.0.0.1\n```\n\nIt will translate `sftp_server` into `127.0.0.1`.\n\nNext edit `docker-compose.yml` and put:\n\n```yml\nversion: '2'\nservices:\n  dns:\n    restart: always\n    image: strm/dnsmasq\n    volumes:\n      - ./dnsmasq.conf:/etc/dnsmasq.conf\n    ports:\n      - \"53:53/udp\"\n    cap_add:\n      - NET_ADMIN\n  sftp:\n    image: atmoz/sftp\n    ports:\n      - \"22:22\"\n    command: foo:pass:::upload\n```\n\nNext launch dnsmasq and sftp:\n\n```bash\ndocker-compose up\n```\nCredentials to local SFTP:\n- user: foo\n- password: pass\n\n### 2. Configure `ftts-payments-sap-sftp-push` application\n\nRun `npm run copy-config`\n\nThis config will point to local SFTP and DNS.\n\nIt will configure application with local blob storage, blob name: `bbb`.\n\n### 3. Run application\n\n```bash\nnpm install\nnpm run start\n```\n\n### 4. Put file to Azure blob\n\nLaunch Microsoft Azure Storage Explorer/Azurite, connect to local storage, create `bbb` blob and upload some file.\n\nAlternatively if you have trouble trying to use Storage Explorer/Azurite locally, you can just use the deployed `sapfilestor` (via sap-release-pipeline), create a new container `bbb` and upload your file there. (And point the local.settings.json connection string to the deployed blob store.)\n\n### 5. Within one minute new file should appear in your local SFTP\n\nCheck on Mac OS:\n\n```bash\nsftp foo@localhost\n```\n\nPassword = `pass`\n\nIn case of `Host key verification failed` you can remove old fingerprint by\n\n```bash\nssh-keygen -R localhost\n```\n\nAfter connection, in `sftp` console cd to the `upload` folder:\n\n```bash\nls -l\n```\n\nUploaded file should be listed.\n\nTo force reprocessing of a blob, delete the blob receipt for that blob from the azure-webjobs-hosts container manually. While reprocessing might not occur immediately, it's guaranteed to occur at a later point in time. To reprocess immediately, the scaninfo blob in azure-webjobs-hosts/blobscaninfo can be updated. Any blobs with a last modified timestamp after the LatestScan property will be scanned again.\n\nDetailed documentation - https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-storage-blob-trigger?tabs=csharp#blob-receipts\n\n### 6. Failed Upload Notifier\n\nIn terms of error, Azure will retry based on maxDequeueCount parameter in the host.json file. \nAfter that the message will be moved to the poison queue 'webjobs-blobtrigger-poison' and Failed Upload Notifier function will be triggered to log proper event.\n\n## Roles validation\n\nRoles validation is enabled by default on all environmemnts. It can be disabled by setting ROLES_VALIDATION environment variable (boolean).\n\n## Healthcheck HTTP endpoint\n\nPayments SAP SFTP Push healthcheck function is a troubleshooting/support function to check connectivity with specific components used by application\n\nGET \u003cpayments-sap-sftp-push-url\u003e/api/\u003cversion\u003e/healthcheck - e.g. /api/v1/healthcheck\n\nResponses:\n\n- HTTP 200 (connections OK)\n\n- HTTP 503 with response body containing specific errors details:\n\n```json\n{\n  \"status\": \"Service unavailable\",\n  \"errors\": [\n    {\n      \"component\": \"\u003cCOMPONENT_NAME\u003e\",\n      \"message\": \"\u003cERROR_MESSAGE\u003e\",\n    }\n  ]\n}\n```\n\n- HTTP 500 with response body containing error message\n\nDocumentation - https://wiki.dvsacloud.uk/pages/viewpage.action?spaceKey=FB\u0026title=Health+Checks","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvsa%2Fftts-payments-sap-sftp-push","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdvsa%2Fftts-payments-sap-sftp-push","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdvsa%2Fftts-payments-sap-sftp-push/lists"}