{"id":19623661,"url":"https://github.com/flowcore-io/nodejs-typescript-transformer-example","last_synced_at":"2025-10-26T09:37:39.503Z","repository":{"id":191682047,"uuid":"685165813","full_name":"flowcore-io/nodejs-typescript-transformer-example","owner":"flowcore-io","description":"An example typescript transformer that can be forked and used on the Flowcore platform","archived":false,"fork":false,"pushed_at":"2024-02-09T13:36:33.000Z","size":1479,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-01-09T11:54:49.931Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flowcore-io.png","metadata":{"files":{"readme":".github/readme/create-repo-from-template.gif","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-30T16:46:39.000Z","updated_at":"2023-09-08T14:08:18.000Z","dependencies_parsed_at":"2023-08-31T05:00:27.963Z","dependency_job_id":"fac5dc05-ad64-4924-8cc4-b28c8b2bc8bd","html_url":"https://github.com/flowcore-io/nodejs-typescript-transformer-example","commit_stats":null,"previous_names":["flowcore-io/nodejs-typescript-transformer-example"],"tags_count":12,"template":true,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flowcore-io%2Fnodejs-typescript-transformer-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flowcore-io%2Fnodejs-typescript-transformer-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flowcore-io%2Fnodejs-typescript-transformer-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flowcore-io%2Fnodejs-typescript-transformer-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flowcore-io","download_url":"https://codeload.github.com/flowcore-io/nodejs-typescript-transformer-example/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240921292,"owners_count":19878912,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T11:35:02.195Z","updated_at":"2025-10-15T06:59:05.797Z","avatar_url":"https://github.com/flowcore-io.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# NodeJS Typescript Transformer Example Repository\n\nThis repository contains a simple example of a NodeJS Typescript Transformer.\n\n## Entrypoint\n\nThe `main.ts` file is the entrypoint for the transformer. It contains the following endpoints:\n\n### GET /health\n\nThis endpoint is used by the transformer shell to check if the transformer is healthy.\n\n### POST /transform\n\nThis endpoint is used by the transformer shell to transform data.\n\n## 📝 Quick start\n\n### 📑 Generate your own repository from this template\n\n\u003cimg alt=\"Create a new repo from the template\" src=\"./.github/readme/create-repo-from-template.gif\" width=\"900\" /\u003e\n\u003cbr/\u003e\n\n## 🚀 Deployment\n\nWe try to simplify your development experience, by including a pipeline that will automatically test, build and push the\nrelease artifact to the GitHub release. However, if you wish to modify the GitHub Action workflow, or use your own, you\nare completely free to do so.\n\n### 🔧 Preparing our pre-made Pipeline\n\nBefore you can utilise our pre-built pipeline, you need to authenticate yourself with it. This is done by:\n\n1. generating a personal access token with the permission of creating artifacts.\n2. adding a `new repository secret` to your `Settings \u003e Secrets and variables \u003e Actions`, with the name\n   of `RELEASE_GITHUB_TOKEN`, and the value of your newly created personal token\n\nYou can follow our step-by-step video guide, if you are not accustomed to this process:\n\n[Automating the release process for your Transformer with GitHub Actions](https://youtu.be/QveUgYLJWe8)\n\n### 1️⃣.2️⃣.0️⃣ Release Artifact Versioning\n\nWe utilise [Conventional Commits](https://www.conventionalcommits.org/en/v1.0.0/#summary) to automatically manage the\nversioning of your released artifacts. In short:\n\n- prefixing your commits with `feat: ` will trigger a minor version (1.**2**.0)\n- prefixing your commits with `fix: ` will trigger a patch version (1.0.**3**)\n\nAs an example: `feat: splitting email into username and domain`, or `fix: email now splits correctly by @ symbol`\n\nIf you need anything more granular, then you can refer to the link above.\n\n### 🔄 Release Process\n\nIf using the correct [conventional commit syntax](#120-release-artifact-versioning), then the release process is as\nfollows:\n\nAnything that gets pushed to the `main` branch, will trigger a pull request; that runs tests to validate the release.\nOnce the pull request has been merged the release will be published, together with the artifact.\n\n## 💻 Development\n\nWe have **2 recommended methods** of working on your transformer.\n\n1. [Develop it directly from GitHub](#option-1-develop-directly-in-github-simple)\n2. [Run it locally]()\n\nYour choice may vary depending on your development environment, preference and/or complexity.\n\n### Option 1: Develop directly in GitHub (Simple)\n\n\u003e You can look at how you start the editor directly in your browser [here](https://github.com/github/dev).\n\nMake your changes\nand [push them to your repository](https://code.visualstudio.com/docs/sourcecontrol/intro-to-git#_staging-and-committing-code-changes).\n**Bear in mind** that you need to use conventional commits to be able to release your transformer. You can read more\nabout\nit under [Deployment](#-deployment).\n\n### Option B: Develop locally (Advanced)\n\n### 1. ⚙️ Installation️\n\n\u003e Prerequisites:\n\u003e - NodeJS\n\u003e - Docker\n\u003e\n\u003e And you must clone the project on to your machine\n\ninstall dependencies:\n\n```bash\nyarn install \u0026\u0026 yarn build\n```\n\nrun the transformer shell\n\n```bash\ndocker-compose -f test/docker/docker-compose.yaml up -d\n```\n\nCreate a .env file with the following content:\n\n```bash\nHOST_ADDRESS=host.docker.internal\n```\n\n### 2. 💻 Development\n\nTo start developing with watch mode run:\n\n```bash\nyarn build:watch\n```\n\nIn another terminal or tab, run:\n\n```bash\nyarn test:watch\n```\n\nto run the tests on the built transformer.\n\nWhen changes are made any of the files the transformer will be reloaded and the tests will be run again.\n\n\u003e **Note**: The dist directory needs to be writable by the transformer shell.\n\n## 🔎 Development Overview\n\n### Change the transformer\n\nTo change the transformer, edit the `transform.entrypoint.ts` file. To add functionality on startup edit\nthe `start.entrypoint.ts` file. To add additional health checks edit the `health.entrypoint.ts` file.\n\n### Change the input and output data\n\nTo change the validation of inputs and outputs edit the `test/expected.json` file. This file specify the event payloads\nthat are sent to the transformer and the expected output. The `:uuid:` and `:date:` values for the expected outcome\nmatches to any string.\n\n### Further customization\n\nChange the `test/app.spec.ts` file to add additional tests and more advanced validation. Further change the files in\nthe `src` directory to add more advanced logic.\n\n## 🌕 Utilise your Transformer in Flowcore\n\n\u003c!-- todo: move this to a another part --\u003e\nTo use this transformer in the [Flowcore](https://flowcore.io) platform, create a new adapter and point it to the github\nrelease artifact.\n\nThe shell will then download the artifact, run it and for each data point post to the `transform` endpoint.\n\n\u003c!-- todo: add a video --\u003e\n\n## Migrate from v1 to v2\n\nThe v2 of the transformer shell work differently from v1. The main difference is that the v2 shell loads from a config file rather than through en endpoint. This means that the transformer shell needs to be updated to use the new config file.\n\n### 1. Update the transformer shell docker compose file\n\nchange the `transformer` service in the `docker-compose.yaml` file to use the new image.\n\n```yaml\nversion: \"3.8\"\n\nservices:\n   shell:\n      container_name: transformer-shell\n      image: flowcoreio/adapter-nodejs-transformer-shell:2.1.0\n      ports:\n         - \"3001:3001\"\n         - \"10000:10000\"\n      environment:\n         LOG_LEVEL: debug\n         LOG_PRETTY_PRINT: \"true\"\n         TRANSFORMERS: node\n         PORT: 3001\n         TRANSFORMER_DEV_MODE: \"true\"\n      volumes:\n         - ./../../dist:/app/transformers/test-transformer\n         - ./../config:/usr/src/app/transformer\n```\n\nadd a new transformer.json configuration file at `test/config/transformer.json` with the following content:\n\n```json\n{\n   \"name\": \"test-transformer\",\n   \"version\": \"1.0.0\",\n   \"runtime\": \"node\",\n   \"artifactUrl\": \"/app/transformers/test-transformer\",\n   \"entrypoint\": \"main.js\",\n   \"startTimeTimeout\": 10000,\n   \"port\": 10000\n}\n```\n\nthen finally update the app.spec.ts file to use this new setup.\n\n1. remove the TRANSFORMER_BLUEPRINT const\n\n```typescript\nconst TRANSFORMER_BLUEPRINT: TransformerBlueprint = JSON.parse(fs.readFileSync(path.join(process.cwd(), \"test/config\", \"transformer.json\"), \"utf-8\"));\n```\n\n2. change the check to use the shell health endpoint in `beforeAll`\n\nadd adapter port const\n\n```typescript\nconst adapterPort = 3001;\n```\n\nreplace the try catch block that loads the transformer with the following code:\n\n```typescript\nawait waitForExpect(async () =\u003e {\nconsole.debug(`Checking if transformer is loaded on http://localhost:${adapterPort}/health`);\nconst axiosResponse = await axios.get(\n        `http://localhost:${adapterPort}/health`,\n);\n\nif (axiosResponse.status !== 200) {\n   console.debug(`Transformer not loaded on http://localhost:${adapterPort}/health`, axiosResponse.data);\n}\n\nexpect(axiosResponse.status).toEqual(200);\n}, 10000, 1000);\n```\n\n3. remove all references to processId\n\nremove the `processId` let variable and the `processId` from the `axios.post` call. This is the simplified version of the `axios.post` call:\n\n```typescript\nconst processedResult = await axios.post(\n        \"http://localhost:3001/transform\",\n        data,\n);\n```\n\n4. and finally remove the unload transformer call in `afterAll`, it should just look like this:\n\n```typescript\nafterAll(async () =\u003e {\n    server.close();\n  });\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowcore-io%2Fnodejs-typescript-transformer-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflowcore-io%2Fnodejs-typescript-transformer-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflowcore-io%2Fnodejs-typescript-transformer-example/lists"}