{"id":18463833,"url":"https://github.com/smartive/proc-that","last_synced_at":"2025-04-06T01:09:56.122Z","repository":{"id":57330806,"uuid":"54781339","full_name":"smartive/proc-that","owner":"smartive","description":"proc(ess)-that - easy extendable ETL tool for Node.js. Written in TypeScript.","archived":false,"fork":false,"pushed_at":"2025-03-11T23:43:52.000Z","size":423,"stargazers_count":39,"open_issues_count":8,"forks_count":6,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-30T00:06:59.489Z","etag":null,"topics":["etl","extractor","loader","node","observables","rxjs","transformer","typescript"],"latest_commit_sha":null,"homepage":"https://smartive.github.io/proc-that/","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/smartive.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-03-26T13:54:39.000Z","updated_at":"2025-02-04T18:53:44.000Z","dependencies_parsed_at":"2024-06-19T05:15:40.281Z","dependency_job_id":"795953f7-7353-4c00-ae61-f78f346da345","html_url":"https://github.com/smartive/proc-that","commit_stats":{"total_commits":109,"total_committers":7,"mean_commits":"15.571428571428571","dds":"0.27522935779816515","last_synced_commit":"8a85364b9bef3ac322e0bfe1eeabfeb23a344a09"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartive%2Fproc-that","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartive%2Fproc-that/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartive%2Fproc-that/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/smartive%2Fproc-that/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/smartive","download_url":"https://codeload.github.com/smartive/proc-that/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419861,"owners_count":20936012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etl","extractor","loader","node","observables","rxjs","transformer","typescript"],"created_at":"2024-11-06T09:08:06.583Z","updated_at":"2025-04-06T01:09:56.094Z","avatar_url":"https://github.com/smartive.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# proc-that\n\nproc(ess)-that - easy extendable etl tool for nodejs written in typescript.\n\nBasically instantiate the `Etl` class and add extractors (which pull data from a datasource), transformers (which process the extracted data) and loaders (they load the results into a sink).\n\nA basic, hypothetic example could be: \"Load data from a JSON array, snake_case all properties and store those objects into a mongoDB.\"\n\nThe package is written in `typescript` but can be used in plain javascript as well\n\n##### A bunch of badges\n\n[![Build Status](https://travis-ci.org/smartive/proc-that.svg?maxAge=3600)](https://travis-ci.org/smartive/proc-that)\n[![Build Status](https://ci.appveyor.com/api/projects/status/wm7ydpf62e9518h8?svg=true)](https://ci.appveyor.com/project/buehler/proc-that)\n[![npm](https://img.shields.io/npm/v/proc-that.svg?maxAge=3600)](https://www.npmjs.com/package/proc-that)\n[![Coverage status](https://img.shields.io/coveralls/smartive/proc-that.svg?maxAge=3600)](https://coveralls.io/github/smartive/proc-that)\n[![license](https://img.shields.io/github/license/smartive/proc-that.svg?maxAge=2592000)](https://github.com/smartive/proc-that)\n[![semantic-release](https://img.shields.io/badge/%20%20%F0%9F%93%A6%F0%9F%9A%80-semantic--release-e10079.svg)](https://github.com/semantic-release/semantic-release)\n[![Greenkeeper badge](https://badges.greenkeeper.io/smartive/proc-that.svg)](https://greenkeeper.io/)\n\n## Usage\n\n```typescript\nimport { Etl } from \"proc-that\";\n\nnew Etl()\n  .addExtractor(/* class that implements Extractor */)\n  .addTransformer(/* class that implements Transformer */)\n  .addLoader(/* class that implements Loader */)\n  .start()\n  .subscribe(progress, error, success);\n```\n\nAfter all objects are extracted, transformed and loaded, the `.start()` observable completes and the process is finished.\n\nBelow is a list if extractors and loaders that are already implemented. Feel free to implement your own extractor / transformer / loader and contribute it to this list with a PR.\n\n## Extractors\n\n| Name                       | Description                       | Link                                                 |\n| -------------------------- | --------------------------------- | ---------------------------------------------------- |\n| `proc-that-rest-extractor` | Extract objects from GET requests | https://github.com/smartive/proc-that-rest-extractor |\n\n## Loaders\n\n| Name                       | Description                                 | Link                                                 |\n| -------------------------- | ------------------------------------------- | ---------------------------------------------------- |\n| `proc-that-elastic-loader` | Load transformed objects into elasticsearch | https://github.com/smartive/proc-that-elastic-loader |\n\n## Implement your own\n\nTo ease up implementing your own extractors / transformers or loaders, just create a new repository and install `proc-that` as a dev-dependency. This package contains the needed definition files for the interfaces you need to create the extensions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartive%2Fproc-that","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsmartive%2Fproc-that","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsmartive%2Fproc-that/lists"}