{"id":45491440,"url":"https://github.com/floherent/node-transform-builder","last_synced_at":"2026-02-22T17:00:57.541Z","repository":{"id":333760553,"uuid":"1070832936","full_name":"floherent/node-transform-builder","owner":"floherent","description":"Build Coherent Spark's Node Transforms","archived":false,"fork":false,"pushed_at":"2026-01-20T21:45:27.000Z","size":42,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-01-21T07:15:59.659Z","etag":null,"topics":["coherent-spark","jsonata","node-transforms"],"latest_commit_sha":null,"homepage":"https://docs.coherent.global","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/floherent.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-06T13:51:30.000Z","updated_at":"2026-01-20T21:45:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/floherent/node-transform-builder","commit_stats":null,"previous_names":["floherent/node-transform-builder"],"tags_count":null,"template":true,"template_full_name":null,"purl":"pkg:github/floherent/node-transform-builder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/floherent%2Fnode-transform-builder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/floherent%2Fnode-transform-builder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/floherent%2Fnode-transform-builder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/floherent%2Fnode-transform-builder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/floherent","download_url":"https://codeload.github.com/floherent/node-transform-builder/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/floherent%2Fnode-transform-builder/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29719280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-22T15:10:41.462Z","status":"ssl_error","status_checked_at":"2026-02-22T15:10:04.636Z","response_time":110,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coherent-spark","jsonata","node-transforms"],"created_at":"2026-02-22T17:00:38.211Z","updated_at":"2026-02-22T17:00:57.530Z","avatar_url":"https://github.com/floherent.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Node Transform Document Builder\n\nThis is a specialized tool for building [Node transform documents][node-transform-docs] designed for the Coherent platform.\n\nThink of this as your go-to solution when standard JSONata transforms aren't quite enough for your needs.\nIt's designed for those moments when you need to do more than simple data mapping — e.g., implementing\n[MapReduce][map-reduce] patterns, orchestrating multi-step business logic, or processing complex\nunstructured data that requires a mix of [JSONata][jsonata] expressions (or [fast-xml-parser][fast-xml-parser]\nfor XML data), and custom JavaScript logic. Plus, it easily integrates with Coherent Spark's\n[Execute API (v3)][exec-v3] to help you build sophisticated data transformation pipelines.\n\n\u003e [!IMPORTANT]\n\u003e This tool is here to give you a solid foundation for building Node.js transform documents\n\u003e with JSONata workflows, but it's not a complete out-of-the-box solution. You may think of it as getting\n\u003e you about 80% of the way there; the final 20% will depend on your specific data contract and how your\n\u003e business logic flows. If you're tackling something more complex, consider this a great starting point\n\u003e that you can build upon and customize to fit your exact needs.\n\n## TL;DR\n\nThis repository helps you create a production-ready [handler](src/handler.ts) — a Node transform document\nin JSON format that you can deploy to Coherent Spark. It does this by combining JSONata expressions,\nJavaScript code, and Coherent Spark's [Execute API (v3)][exec-v3] into a single, deployable JSON file.\n\nTo get you started quickly, the build scripts will generate a sample transform document in the `dist`\ndirectory. This sample demonstrates a \"volume of a cylinder\" calculation, so you'll want to customize\nthe [request.jsonata](transforms/request.jsonata) and [response.jsonata](transforms/response.jsonata)\nfiles to match your own data contract and business logic. The sample data in the [assets](./assets/)\nfolder follows the same example. Oh, and don't forget to create a `.env` file (you can use\n[.env.local](.env.local) as a template) with the environment variables you'll need for building and\ndeploying your transform.\n\nThen, using [Node.js 22](https://nodejs.org/en/download), run the following commands:\n\n```bash\n# Install dependencies (run this only the first time)\nnpm install\n\n# Build the transform document\nnpm run build\n\n# Deploy the transform document (run this only if you need to upload it to Coherent)\nnpm run deploy\n```\n\n\u003e [!TIP]\n\u003e To deploy transform documents to Coherent, you need to have valid credentials for the Coherent platform\n\u003e (i.e., Spark base URL and access token), or use Spark UI to upload the transform document.\n\n## Table of Contents\n\n- [Getting Started](#getting-started)\n- [Prerequisites](#prerequisites)\n- [Installation](#installation)\n- [Usage](#usage)\n- [Workflow](#workflow)\n- [Limitations](#limitations)\n\n## Getting Started\n\nThe tool is designed to help you handle complex unstructured data transformations with ease by:\n\n- transforming data using JSONata expressions for request/response mapping\n- executing Excel-based business logic via Coherent Spark's Execute API (v3)\n\nBut its real power is the scaffolding it provides for building complex data transformation pipelines.\nIt takes into account different stages of the build process, including request/response mapping,\ndata validation, and error handling. So, if you need to build a complex data transformation pipeline,\nmake the appropriate changes to the [handler](src/handler.ts), [utils](src/utils.ts) files, and\nJSONata expressions, and you'll be good to go.\n\n## Prerequisites\n\nBefore getting started, confirm that you have the following:\n\n- [Node.js](https://nodejs.org/) (version 22 or higher)\n- Valid credentials for Spark execution environment, including tenant, folder, and service names.\n- Access to deploy and manage [transform documents](https://docs.coherent.global/spark-apis/transforms-api/transform-documents-apis).\n\n## Installation\n\n- Clone this repository\n\n```bash\n  git clone \u003crepository-url\u003e\n  cd node-transform-builder\n```\n\n- Install dependencies\n\n```bash\nnpm install\n```\n\n- Run the test suite to ensure everything is working:\n\n```bash\nnpm test \u0026\u0026 npm run test:e2e\n```\n\n- Create a `.env` file in the project root with appropriate settings:\n\n```env\nTRANSFORM_MODE=\"Node\" # node | jsonata\nTRANSFORM_NAME=\"my-transform-name\"\nTRANSFORM_EXTERNAL_DOMAINS=\"*.coherent.global,*.googleapis.com\"\nCSPARK_BASE_URL=https://spark.my-env.coherent.global/my-tenant-name\nCSPARK_BEARER_TOKEN=Bearer \u003cmy-access-token\u003e\n```\n\n\u003e [!NOTE]\n\u003e `CSPARK_BASE_URL` and `CSPARK_BEARER_TOKEN` are useful for publishing transform\n\u003e documents to the Coherent platform since part of the deployment process relies on the\n\u003e [Coherent Spark TypeScript SDK][cspark-sdk].\n\n## Usage\n\n### Basic Usage\n\nThe primary workflow involves building transform documents (and optionally deploying them):\n\n```bash\n# Build the project and create transform documents\nnpm run build\n\n# Run the example handler\nnpm run example\n\n# Format code\nnpm run format\n```\n\n### Creating Transform Documents\n\nTransform documents are automatically generated during the build process:\n\n1. `request.jsonata` and `response.jsonata` are validated\n2. TypeScript is compiled to JavaScript (ES Module format)\n3. Transform documents are created in the `dist/` directory\n\n## Workflow\n\n### 1. Architecture and Pipeline\n\n```mermaid\nflowchart TD\n    A([\"Start\"]) --\u003e B[\"**prebuild**:\u003cbr\u003eLint and test JSONata\"]\n    B --\u003e C[\"**build**:\u003cbr\u003eBundle with Rollup\"] \u0026 H[\"Lint JSONata files:\u003cbr\u003e *.jsonata\"]\n    C --\u003e D[\"**postbuild**:\u003cbr\u003eCreate Transform Document\"] \u0026 I[\"Combine into one ESM: \u003cbr\u003e Handler + Utils + Constants\"]\n    H --\u003e L[\"Test JSONata files:\u003cbr\u003e *.jsonata\"]\n    L --\u003e F[\"**dist** Folder:\u003cbr\u003e handler.js + request.jsonata + response.jsonata\"]\n    I --\u003e F\n    F --\u003e J[\"Assemble JSON file: \u003cbr\u003eInject JSONatas, Convert code to CJS, and Compress final content.\"]\n    D --\u003e J\n    J --\u003e G[\"**dist** Folder: *_transform.json\"]\n    G --\u003e E([\"End\"])\n\n    style A fill:#BBDEFB\n    style B color:#FF6D00\n    style C color:#FF6D00\n    style D color:#FF6D00\n    style F fill:#FFF3E0\n    style G fill:#C8E6C9\n    style E fill:#BBDEFB\n```\n\n### 2. Build Process\n\nThe build process consists of several automated steps:\n\n1. **prebuild** - JSONata expression validation and testing\n2. **build** - TypeScript compilation using Rollup\n3. **postbuild** - Transform document generation\n\n### 3. JSONata Processing\n\nThe system uses two main JSONata expressions:\n\n- **Request Transform** (`transforms/request.jsonata`): maps unstructured input data to Spark-compatible format\n- **Response Transform** (`transforms/response.jsonata`): maps Spark response back to expected output format\n\n### 4. Error Handling\n\nThe system implements comprehensive error handling:\n\n- **Compilation Errors**: invalid JSONata expressions are caught during build\n- **Runtime Errors**: API failures are handled gracefully with structured error responses\n- **Validation Errors**: input data validation with meaningful error messages\n\n## Limitations\n\n- No batch or asynchronous processing capabilities (i.e., the tool is synchronous).\n- Large payloads may hit API timeout limits (i.e., 60 seconds).\n- Very complex JSONata transformations may impact performance (less than 6 MB payload).\n- Fast XML parsing is considered but not in use in the current version of the build process.\n- Field Mapping requires exact field name matching for proper transformations.\n\n## Copyright and License\n\n[Apache-2.0](LICENSE)\n\n\u003c!-- References --\u003e\n\n[node-transform-docs]: https://docs.coherent.global/spark-apis/transforms-api/transform-types/nodejs22\n[map-reduce]: https://en.wikipedia.org/wiki/MapReduce\n[exec-v3]: https://docs.coherent.global/spark-apis/execute-api/execute-api-v3\n[fast-xml-parser]: https://www.npmjs.com/package/fast-xml-parser\n[cspark-sdk]: https://www.npmjs.com/package/@cspark/sdk\n[jsonata]: https://jsonata.org/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffloherent%2Fnode-transform-builder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffloherent%2Fnode-transform-builder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffloherent%2Fnode-transform-builder/lists"}