{"id":15474841,"url":"https://github.com/top-guns/etl-gun","last_synced_at":"2025-04-22T14:10:26.392Z","repository":{"id":59250902,"uuid":"536169691","full_name":"top-guns/etl-gun","owner":"top-guns","description":"᠊ᡃ࡚ࠢ࠘ ⸝່ࠡࠣ᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊ࠢ࠘᠊᠊° ̿̿ ̿̿💥  ETL toolkit which supports generators, RxJS streams, IxJS iterables, nodejs streams, error handling, business rules and many more","archived":false,"fork":false,"pushed_at":"2024-01-08T13:04:57.000Z","size":2151,"stargazers_count":20,"open_issues_count":1,"forks_count":1,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-22T05:58:56.126Z","etag":null,"topics":["cockroachdb","csv","etl","ftp","javascript","json","magento","mariadb","mssql","mysql","nodejs","oracle","postgresql","redshift","sqlite","telegram","trello","typescript","webdav","xml"],"latest_commit_sha":null,"homepage":"https://github.com/top-guns/etl-gun","language":"TypeScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/top-guns.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-09-13T14:37:22.000Z","updated_at":"2025-03-13T09:50:18.000Z","dependencies_parsed_at":"2024-11-14T13:21:53.846Z","dependency_job_id":"19724d3d-c6aa-48a4-9809-45cd1a585d04","html_url":"https://github.com/top-guns/etl-gun","commit_stats":{"total_commits":547,"total_committers":3,"mean_commits":"182.33333333333334","dds":0.009140767824497242,"last_synced_commit":"269fa567dfd3877350a741723e2a5cfe666aff17"},"previous_names":["igor-berezhnoy/rxjs-etl-kit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/top-guns%2Fetl-gun","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/top-guns%2Fetl-gun/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/top-guns%2Fetl-gun/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/top-guns%2Fetl-gun/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/top-guns","download_url":"https://codeload.github.com/top-guns/etl-gun/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250255700,"owners_count":21400410,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cockroachdb","csv","etl","ftp","javascript","json","magento","mariadb","mssql","mysql","nodejs","oracle","postgresql","redshift","sqlite","telegram","trello","typescript","webdav","xml"],"created_at":"2024-10-02T03:05:04.331Z","updated_at":"2025-04-22T14:10:26.370Z","avatar_url":"https://github.com/top-guns.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1\u003e\nETL-Gun\n\u003cimg src=\"./static/gun-02.png\" height=\"40px\" style=\"margin-left: 10px; vertical-align: middle\"/\u003e\n\u003c/h1\u003e\n\n\u003c!-- ![Typescript](https://badgen.net/badge/icon/typescript?icon=typescript\u0026label) --\u003e\n[![NPM Version][npm-image]][npm-url]\n[![NPM Downloads][downloads-image]][downloads-url]\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Build Status](https://github.com/top-guns/etl-gun/actions/workflows/project-ci.yml/badge.svg?branch=main)](https://github.com/top-guns/etl-gun/actions?query=branch%3Amain+workflow%3A\"Project%20CI\")\n[![codecov](https://codecov.io/gh/top-guns/etl-gun/branch/main/graph/badge.svg?token=SPF3FMSOID)](https://codecov.io/gh/top-guns/etl-gun)\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun?ref=badge_shield)\n\u003c!-- [![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun.svg?type=small)](https://app.fossa.com/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun?ref=badge_small) --\u003e\n\u003c!-- [![npm package](https://nodei.co/npm/etl-gun.png?mini=true)](https://nodei.co/npm/etl-gun/) --\u003e\n\nETL-Gun is a platform that employs promises, generators, IxJs iterables, RxJs observables and nodejs streams, allowing developers to build stream-based ETL (Extract, Transform, Load) pipelines complete with buffering, bulk operations, error handling and many useful features.\n\n[//]: # (https://img.shields.io/codecov/c/github/top-guns/etl-gun/.svg   https://codecov.io/gh/top-guns/etl-gun)\n\n[logo]: https://img.shields.io/badge/©%20Powered%20by-TOP%20Guns%20︻デ═一-white?color=black\u0026labelColor=white\n[logo2]: https://img.shields.io/badge/Powered%20by%20©-TOP%20Guns%20︻デ═一-white?logo=nodedotjs\u0026logoColor=white\n[logo1]: https://badgen.net/badge/Powered%20by/ETL%20Gun%20%EF%B8%BB%E3%83%87%E2%95%90%E4%B8%80-/blue?icon=github\n[top-guns-url]: https://github.com/top-guns\n[npm-image]: https://img.shields.io/npm/v/etl-gun.svg\n[npm-url]: https://npmjs.org/package/etl-gun\n[downloads-image]: https://img.shields.io/npm/dm/etl-gun.svg\n[downloads-url]: https://npmjs.org/package/etl-gun\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n\u003cimg src=\"./static/ETL.png\" alt=\"Diagram\" title=\"Logo\" style=\"max-width: 100%;\"\u003e\n\n\u003cdiv style=\"text-align: right\"\u003e\n\n[![TOP Guns][logo]][top-guns-url]\n\n\u003c/div\u003e\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Table of Contents\n* [Why / when would I need this?](#why--when-would-i-need-this)\n* [Installation](#installation)\n* [Usage](#usage)\n* [Concept](#concept)\n* [Features](#features)\n* [GUI](#gui)\n* [Examples (how to)](#examples-how-to)\n    * [Export rows from Postgres table to csv-file (postgresql -\u003e .csv)](#export-rows-from-postgres-table-to-csv-file-postgresql---csv)\n    * [Sort rows in csv-file by the first column (.csv -\u003e .csv)](#sort-rows-in-csv-file-by-the-first-column-csv---csv)\n    * [Create telegram bot with 'echo' functionality](#create-telegram-bot-with-echo-functionality)\n* [API Reference](#api-reference)\n    * [Core classes](#core)\n        * [BaseCollection](#basecollection) - base class for all collections\n        * [Errors](#errors) - queues to store and process errors which occurred in other endpoints\n    * [Endpoints](#endpoints-and-its-collections)\n        * [Auxiliary]()\n            * [Memory](#memory) - contains Buffer and Queue collections to operate data in memory\n            * [Interval](#interval)\n        * [Filesystems]()\n            * [Local filesystem](#local-filesystem)\n            * [FTP, FTPS](#ftp)\n            * [SFTP](#sftp)\n            * [WebDAV](#webdav)\n        * [File formates]()\n            * [Csv](#csv)\n            * [Json](#json)\n            * [Xml](#xml)\n        * [Databases]()\n            * [Knex](#knex)\n            * [CockroachDB](#cockroachdb)\n            * [MariaDb](#mariadb)\n            * [MS SQL Server](#ms-sql-server)\n            * [MySQL](#mysql) - mysql and mysql2 drivers both are supported\n            * [Oracle DB](#oracle-db)\n            * [PostgreSQL](#postgres)\n            * [Amazone Redshift](#amazone-redshift)\n            * [SQLite](#sqlite)\n        * [Messangers]()\n            * [GeneralEmail](#generalemail) - any smtp + imap email server can be used\n            * [Gmail](#gmail)\n            * [sms.ru](#smsru)\n            * [Telegram](#telegram)\n        * [CMS]()\n            * [Magento](#magento)\n        * [Task tracking systems]()\n            * [Trello](#trello)\n            * [Zendesk](#zendesk)\n    * [Operators](#operators)\n        * [run](#run)\n        * [log](#log)\n        * [expect](#expect) - as expect() in unit test engines, used for data validation\n        * [where](#where) - similar to rxjs filter() operator, but more useful to data processing\n        * [collect](#collect) - analog of rxjs buffer operators, but with improvements for data processing\n        * [push](#push)\n        * [rools](#rools) - integration with business rules engine\n        * [move](#move)\n        * [copy](#copy)\n        * [numerate](#numerate)\n        * [addField](#addfield)\n        * [addColumn](#addcolumn)\n        * [join](#join)\n        * [mapAsync](#mapasync) - as rxjs map() operator, but work with async callback handler\n    * [Misc](#misc)\n        * [GoogleTranslateHelper](#googletranslatehelper)\n        * [HttpClientHelper](#httpclienthelper)\n        * [Header](#header)\n        * [Utility functions](#utility-functions) \n- [License](#license)\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Why / when would I need this?\n\n**ETL-Gun** is a simple **ETL glue** represented as an extention to the **RxJs** library. \nTypically, you'd use **ETL-Gun** to help with ETL processes. It can extract data from the one or more sources, transform it and load to one or more destinations in nedded order.\n\nYou can use javascript and typescript with it.\n\n**ETL-Gun** will **NOT** help you with \"big data\" - it executes on the one computer and is not supports clustering from the box.\n\nHere's some ways to use it:\n\n1. Read some data from database and export it to the .csv file and vice versa\n2. Create file converters\n3. Filter or sort content of some files\n4. Run some queries in database\n5. Scan remote ftp folder and download some files from it\n\nYou can find many examples of using **ETL-Gun** in the API Reference section of this file.\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Installation\n\n[![npm package](https://nodei.co/npm/etl-gun.png?downloads=true\u0026downloadRank=true\u0026stars=true)](https://nodei.co/npm/etl-gun/)\n\n```\nnpm install etl-gun\n```\nor\n```\nyarn add etl-gun\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Usage\n\n**Info:** You can get the ready to use blank example project in the [example-project](./examples/example-project/) folder of this repository.\n\n**Info:** You can convert your etl project to the systemd service and setup schedule for it with the [etl-gun-service](./examples/example-project/) tool.\n\n**Warning:** Since the version 2.0.4 this library is native [ESM](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Modules) and no longer provides a CommonJS export. If your project uses CommonJS, you will have to [convert to ESM](https://gist.github.com/sindresorhus/a39789f98801d908bbc7ff3ecc99d99c) or use the [dynamic `import()`](https://v8.dev/features/dynamic-import) function.\n\n\u003cins\u003eIntroductory example of library using: postgresql -\u003e .csv\u003c/ins\u003e\n\n```typescript\nimport { map } from \"rxjs\";\nimport { Csv, GuiManager, Header, Postgres, log, push, run } from \"etl-gun\";\n\n// If you want to view GUI, uncomment the next line of code\n// new GuiManager();\n\n// Step 1: endpoint creation\nconst postgres = new Postgres.Endpoint(\"postgres://user:password@127.0.0.1:5432/database\");\nconst source = postgres.getTable('users');\n\nconst csv = new Csv.Endpoint('./dest-folder');\nconst dest = csv.getFile('users.scv');\n\nconst header = new Header(\"id\", \"name\", \"login\", \"email\");\n\n// Step 2: transformation streams creation\nconst sourceToDest$ = source.selectRx().pipe(\n    log(),\n    map(v =\u003e header.objToArr(v)),\n    push(dest)\n);\n\n// Step 3: runing transformations (and wait until they finish, if necessary)\nawait run(sourceToDest$);\n ```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Concept\n\n**ETL-Gun** contains several main concepts: \n* Endpoints - sources and destinations of data, which holds connection to the one system instance and other parameters of this system, and groups methods to get collections related this system\n* Collections - data object types exists in the endpoint system\n* Piplines (or streams) - routs of data transformation and delivery, based on **RxJs** streams\n\nUsing of this library consists of 3 steps:\n\n1. Define your endpoints and collections for sources and destinations\n2. Define data transformation pipelines using **pipe()** method of input streams of your source endpoints\n3. Run transformation pipelines in order and wait for completion\n\nETL process:\n\n* **Extract**: Data extraction from the source collection performs with **selectRx()** method, which returns the **RxJs** stream\n* **Transform**: Use any **RxJs** and **ETL-Gun** operators inside **pipe()** method of the input stream to transform the input data. To complex data transformation you can use the **Memory.Endpoint** class, which can store data and which collections have **forEach()** and some other methods to manipulate with data in it\n* **Load**: Loading of data to the destination endpoint performs with **push()** collection operator\n\nChaining:\n\nChaning of data transformation performs with **pipe()** method of the input data stream. \nChaning of several streams performs by using **await** with **run()** procedure.\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Features\n\n\u003c!-- ![NodeJS](https://img.shields.io/badge/node.js-6DA55F?style=for-the-badge\u0026logo=node.js\u0026logoColor=white)\n![TypeScript](https://img.shields.io/badge/typescript-%23007ACC.svg?style=for-the-badge\u0026logo=typescript\u0026logoColor=white)\n![JavaScript](https://img.shields.io/badge/javascript-%23323330.svg?style=for-the-badge\u0026logo=javascript\u0026logoColor=%23F7DF1E)\n![Yarn](https://img.shields.io/badge/yarn-%232C8EBB.svg?style=for-the-badge\u0026logo=yarn\u0026logoColor=white)\n![Git](https://img.shields.io/badge/git-%23F05033.svg?style=for-the-badge\u0026logo=git\u0026logoColor=white)\n![Visual Studio Code](https://img.shields.io/badge/Visual%20Studio%20Code-0078d7.svg?style=for-the-badge\u0026logo=visual-studio-code\u0026logoColor=white)\n![Postman](https://img.shields.io/badge/Postman-FF6C37?style=for-the-badge\u0026logo=postman\u0026logoColor=white)\n![macOS](https://img.shields.io/badge/mac%20os-000000?style=for-the-badge\u0026logo=macos\u0026logoColor=F0F0F0)\n![Linux](https://img.shields.io/badge/Linux-FCC624?style=for-the-badge\u0026logo=linux\u0026logoColor=black)\n![Ubuntu](https://img.shields.io/badge/Ubuntu-E95420?style=for-the-badge\u0026logo=ubuntu\u0026logoColor=white)\n![Docker](https://img.shields.io/badge/docker-%230db7ed.svg?style=for-the-badge\u0026logo=docker\u0026logoColor=white)\n![GitHub](https://img.shields.io/badge/github-%23121011.svg?style=for-the-badge\u0026logo=github\u0026logoColor=white) --\u003e\n\n![NodeJS](https://img.shields.io/badge/node.js-6DA55F?style=for-the-badge\u0026logo=node.js\u0026logoColor=white)\n![TypeScript](https://img.shields.io/badge/typescript-%23007ACC.svg?style=for-the-badge\u0026logo=typescript\u0026logoColor=white)\n![JavaScript](https://img.shields.io/badge/javascript-%23323330.svg?style=for-the-badge\u0026logo=javascript\u0026logoColor=%23F7DF1E)\n![RxJS](https://img.shields.io/badge/rxjs-%23B7178C.svg?style=for-the-badge\u0026logo=reactivex\u0026logoColor=white)\n![Postgres](https://img.shields.io/badge/postgres-%23316192.svg?style=for-the-badge\u0026logo=postgresql\u0026logoColor=white)\n![MySQL](https://img.shields.io/badge/mysql-%2300f.svg?style=for-the-badge\u0026logo=mysql\u0026logoColor=white)\n![MariaDB](https://img.shields.io/badge/MariaDB-003545?style=for-the-badge\u0026logo=mariadb\u0026logoColor=white)\n![MicrosoftSQLServer](https://img.shields.io/badge/Ms%20SQL%20Server-CC2927?style=for-the-badge\u0026logo=microsoft%20sql%20server\u0026logoColor=white)\n![Oracle](https://img.shields.io/badge/Oracle-F80000?style=for-the-badge\u0026logo=oracle\u0026logoColor=white)\n![SQLite](https://img.shields.io/badge/sqlite-%2307405e.svg?style=for-the-badge\u0026logo=sqlite\u0026logoColor=white)\n![Trello](https://img.shields.io/badge/Trello-%23026AA7.svg?style=for-the-badge\u0026logo=Trello\u0026logoColor=white)\n![Gmail](https://img.shields.io/badge/Gmail-D14836?style=for-the-badge\u0026logo=gmail\u0026logoColor=white)\n![Telegram](https://img.shields.io/badge/Telegram-2CA5E0?style=for-the-badge\u0026logo=telegram\u0026logoColor=white)\n\n* Simple way to use! Consists of only 3 steps: \n  1. Create endpoints and get all collections which you need \n  2. Create pipelines to process collection data (via **select** method of the source collections)\n  3. Run piplines in order you want (with **run** operator)\n* Support all main data processing paradigms:\n  1. Simple async/await - with select() method which returns promise\n  2. Generators - with selectGen() method which returns generator as AsyncGenerator generic class\n  3. **IxJs** - with selectIx() which returns AsyncIterable generic class fully compatible with **IxJs** library\n  4. **RxJs** - with selectRx() which returns Observable generic class fully compatible with **RxJs** library, it's observables, operators etc.\n  5. Node ReadableStream - with selectStream() which returns ReadableStream generic class\n* This library contains embedded debug console. It created as console application and works in any terminals. It supports step-by-step debuging with watching the processed values. If you want to use this GUI - you simply need to create the instance of **GuiManager** class before any endpoints and collections creation (see [GUI](#gui))\n* Library written in typescript, contains end systems types information and full support of types checking. But you can use it in javascript applications too\n* Contains many kind of sources and destinations, for example many relational databases (Postgre, Mysql, ...), file formats (csv, json, xml), business applications (Magento, Trello, ZenDesk, ...), etc.\n* Work with any types of input/output data, including arrays any hierarchical data structures (json, xml)\n* With endpoint events mechanism you can handle different stream events, for example stream start/end, errors and other (see [Endpoint](#endpoint))\n* Supports validation and error handling mechanism: \n  1. Data validation with [expect](#expect) operator\n  2. Special endpoint type for errors, which base on queue\n  3. Any collections contains property **errors** with endpoint which collect all errors, occurred while collection processing. This endpoint automatic creates when the collection creates, but you can change it's value to collect errors in place of you chois\n  4. Console GUI display all error collections, statistic for it and it's errors\n* Contains some ready to use helpers and integrations, for example you can translate some data to another language with [GoogleTranslateHelper](#googletranslatehelper)\n* Contains business rules integration and allows to extract analisys and transformation logic from the etl program sources, and then change it in runtime without application changing and redeployment (see [rools](#rools))\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# GUI\n\n\u003cimg src=\"https://github.com/top-guns/etl-gun/raw/main/static/GUI.jpg\" alt=\"GUI\" title=\"GUI\" style=\"max-width: 100%\"\u003e\n\n* Simple way to use, you need only create instance of **GuiManager** class before any endpoint creation (at the begin of the program)\n* You can pause the ETL-process and resume it with 'space' on keyboard\n* With 'enter' you can execute ETL process step-by-step in pause mode\n* With 'esc' you can quit the program\n* GUI display full list of created endpoints, collections, their statuses and last values recived from (or pushed to) them\n* Logs are displayed in footer part of console window\n* You can select the log window with 'tab' and scroll it with up/down arrows\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# Examples (how to)\n\n### Export rows from Postgres table to csv-file (postgresql -\u003e .csv)\n\n```typescript\nimport { Postgres, Csv, Header, log, push, run } from \"etl-gun\";\nimport { map } from \"rxjs\";\n\nconst postgres = new Postgres.Endpoint(\"postgres://user:password@127.0.0.1:5432/database\");\nconst source = postgres.getTable('users');\n\nconst csv = new Csv.Endpoint('./dest-folder');\nconst dest = csv.getFile('users.scv');\n\nconst header = new Header(\"id\", \"name\", \"login\", \"email\");\n\nconst sourceToDest$ = source.selectRx().pipe(\n    log(),\n    map(v =\u003e header.objToArr(v)),\n    push(dest)\n);\n\nawait run(sourceToDest$);\n ```\n\n ### Sort rows in csv-file by the first column (.csv -\u003e .csv)\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst csvEndpoint = new etl.Csv.Endpoint();\nconst csv = csvEndpoint.getFile('users.scv');\nconst memory = new etl.Memory.Endpoint();\nconst buffer = memory.getBuffer('buffer 1');\n\nconst scvToBuffer$ = csv.selectRx().pipe(\n    etl.push(buffer)\n);\nconst bufferToCsv$ = buffer.selectRx().pipe(\n    etl.push(csv)\n);\n\nawait etl.run(scvToBuffer$);\n\nbuffer.sort((row1, row2) =\u003e row1[0] \u003e row2[0]);\nawait csv.delete();\n\nawait etl.run(bufferToCsv$)\n ```\n\n ### Create telegram bot with translation functionality\n\n ```typescript\nimport * as etl from \"etl-gun\";\n\nconst telegram = new etl.Telegram.Endpoint();\nconst bot = telegram.startBot('bot 1', process.env.TELEGRAM_BOT_TOKEN!);\nconst translator = new etl.GoogleTranslateHelper(process.env.GOOGLE_CLOUD_API_KEY!, 'en', 'ru');\n\nconst startTelegramBot$ = bot.selectRx().pipe(\n    etl.log(),          // log user messages to the console\n    translator.operator([], [message]), // translate 'message' field\n    etl.push(bot)  // echo input message back to the user\n);\n\netl.run(startTelegramBot$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# API Reference\n\n## Core\n\n### BaseCollection\n\nBase class for all collections. Declares public interface of collection and implements event mechanism.\n\nMethods:\n\n```typescript\n// Read elements from the collection and return it\n// where: condition of the element selection\n// Read elements as promise\nasync select(where?: any): Promise\u003cT[]\u003e;\n// Read elements from the collection and return generator to process it\nasync* selectGen(where?: any): AsyncGenerator\u003cT\u003e;\n// Read elements from the collection and return IxJS iterable to process it\nselectIx(where?: any): AsyncIterable\u003cT\u003e;\n// Read elements from the collection and create RxJS observable to process it\nselectRx(where?: any): Observable\u003cT\u003e;\n// Read elements from the collection and create data stream to process it\nselectStream(where?: any): ReadableStream\u003cT\u003e;\n\n// Add value to the collection (usually to the end of stream)\n// value: what will be added to the collection\nasync insert(value: any);\n\n// Update collection elements\n// where: condition of the element selection\n// value: what will be added to the collection\nasync update(where: any, value: any);\n\n// Clear data of the collection\n// where: condition of the element selection\nasync delete(where?: any);\n\n// Add listener of specified event\n// event: which event we want to listen, see below\n// listener: callback function to handle events\non(event: CollectionEvent, listener: (...data: any[]) =\u003e void);\n\n// Readable/writable property wich contains errors collection instance for this collection\nerrors: Errors.ErrorsQueue;\n```\n\nTypes:\n\n```typescript\nexport type CollectionEvent = \n    \"select.start\" |  // fires at the start of stream\n    \"select.end\" |    // at the end of stream\n    \"select.error\" |  // on error\n    \"select.skip\" |   // when the collection skip some data \n    \"select.up\" |     // when the collection go to the parent element while the tree data processing\n    \"select.down\" |   // when the collection go to the child element while the tree data processing\n    \"recive\" | // for every data value in the stream \n    \"pipe.start\" |    // when processing of any collection element was started\n    \"pipe.end\" |      // when processing of one collection element was ended\n    \"insert\" |        // when data is inserted to the collection\n    \"update\" |        // when data is updated in the collection\n    \"delete\";        // when data is deleted from the collection\n```\n\n## Endpoints and it's collections\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n\n### Errors\n\nStore and process etl errors. Every collection by default has errors property wich contains collection of errors from collection etl process. You can cancel default errors collection creation for any collection, and specify your own manualy created error collection.\n\n#### Endpoint\n\n```typescript\n// Creates new errors collection\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetCollection(collectionName: string, options: CollectionOptions\u003cEtlError\u003e = {}): ErrorsQueue;\n\n// Release errors collection object\n// collectionName: identificator of the releasing collection object\nreleaseCollection(collectionName: string);\n```\n\n#### ErrorsQueue\n\nQueue in memory to store etl errors and process thea. Should be created with **getCollection** method of **Errors.Endpoint**\n\nMethods:\n\n```typescript\n// Create the observable object and send errors data from the queue to it\n// stopOnEmpty: is errors processing will be stopped when the queue is empty\nselectRx(stopOnEmpty: boolean = false): BaseObservable\u003cEtlError\u003e;\n\n// Pushes the error to the queue \n// error: what will be added to the queue\nasync insert(error: EtlError);\n\n// Clear queue\nasync delete();\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Memory\n\nCreate and manipulate with collections of objects in memory.\n\n#### Endpoint\n\n```typescript\n// Creates new memory buffer. This is a generic method so you can specify type of data which will be stored in\n// collectionName: identificator of the creating collection object\n// values: initial data\n// guiOptions: Some options how to display this endpoint\ngetBuffer\u003cT\u003e(collectionName: string, values: T[] = [], guiOptions: CollectionGuiOptions\u003cT\u003e = {}): BufferCollection;\n\n// Release buffer data\n// collectionName: identificator of the releasing collection object\nreleaseBuffer(collectionName: string);\n\ngetQueue\u003cT\u003e(collectionName: string, values: T[] = [], guiOptions: CollectionGuiOptions\u003cT\u003e = {}): QueueCollection;\nreleaseQueue(collectionName: string);\n```\n\n#### BufferCollection\n\nBuffer to store values in memory and perform complex operations on it. Should be created with **getBuffer** method of **MemoryEndpoint**\n\nMethods:\n\n```typescript\n// Create the observable object and send data from the buffer to it\nselectRx(): Observable\u003cT\u003e;\n\n// Pushes the value to the buffer \n// value: what will be added to the buffer\nasync insert(value: T);\n\n// Clear endpoint data buffer\nasync delete();\n\n// Sort buffer data\n// compareFn: You can spacify the comparison function which returns number \n//            (for example () =\u003e v1 - v2, it is behaviour equals to Array.sort())\n//            or which returns boolean (for example () =\u003e v1 \u003e v2)\nsort(compareFn: (v1: T, v2: T) =\u003e number | boolean);\n\n// This function is equals to Array.forEach\nforEach(callbackfn: (value: T, index: number, array: T[]) =\u003e void);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst csvEndpoint = new etl.Csv.Endpoint();\nconst csv = csvEndpoint.getFile('users.scv');\nconst memory = new etl.Memory.Endpoint();\nconst buffer = memory.getBuffer('buffer 1');\n\nconst scvToBuffer$ = csv.selectRx().pipe(\n    etl.push(buffer);\n)\nconst bufferToCsv$ = buffer.selectRx().pipe(\n    etl.push(csv)\n)\n\nawait etl.run(scvToBuffer$);\n\nbuffer.sort((row1, row2) =\u003e row1[0] \u003e row2[0]);\nawait csv.delete();\n\netl.run(bufferToCsv$)\n```\n\n#### QueueCollection\n\nQueue to store values in memory and perform ordered processing of it. Should be created with **getQueue** method of **MemoryEndpoint**\n\nMethods:\n\n```typescript\n// Create the observable object wich send process queue elements one by one and remove processed element from queue\n// dontStopOnEmpty - do we need stop queue processing (unsubscribe) when the queue will be empty\n// interval - pause between elements processing, in milliseconds\nselectRx(dontStopOnEmpty: boolean = false, interval: number = 0): Observable\u003cT\u003e;\n\n// Pushes the value to the queue \n// value: what will be added to the queue\nasync insert(value: T);\n\n// Clear queue\nasync delete();\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Local Filesystem\n\nSearch for files and folders with standart unix shell wildcards [see glob documentation for details](https://www.npmjs.com/package/glob).\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// rootFolder: full or relative path to the folder of intetest\nconstructor(rootFolder: string);\n\n// Creates new FilesystemCollection\n// folderName: subfolder of the root folder and identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetFolder(folderName: string = '.', guiOptions: CollectionGuiOptions\u003cPathDetails\u003e = {}): Collection;\n\n// Release FilesystemCollection\n// folderName: identificator of the releasing collection object\nreleaseFolder(folderName: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Create the observable object and send files and folders information to it\n// mask: search path mask in glob format (see glob documentation)\n//       for example:\n//       *.js - all js files in root folder\n//       **/*.png - all png files in root folder and subfolders\n// options: Search options, see below\nselectRx(mask: string = '*', options?: ReadOptions): BaseObservable\u003cPathDetails\u003e;\n\n// Create folder or file\n// pathDetails: Information about path, which returns from selectRx() method\n// filePath: File or folder path\n// isFolder: Is it file or folder\n// data: What will be added to the file, if it is a file, ignore for folders\nasync insert(pathDetails: PathDetails, data?: string | NodeJS.ArrayBufferView | Iterable\u003cstring | NodeJS.ArrayBufferView\u003e | AsyncIterable\u003cstring | NodeJS.ArrayBufferView\u003e | internal.Stream);\nasync insert(filePath: string, data?: string | NodeJS.ArrayBufferView | Iterable\u003cstring | NodeJS.ArrayBufferView\u003e | AsyncIterable\u003cstring | NodeJS.ArrayBufferView\u003e | internal.Stream, isFolder?: boolean);\n\n// Clear the root folder by mask\n// mask: Which files and folders we need to delete\n// options: Search options, see below \n//          IMPORTANT! Be careful with option includeRootDir because if it is true, and the objectsToSearch is not 'filesOnly',\n//          then the root folder will be deleted with all its content! Including folder itself.\nasync delete(mask: string = '*', options?: ReadOptions);\n```\n\nTypes:\n\n```typescript\ntype ReadOptions = {\n    includeRootDir?: boolean;   // Is root folder itself will be included to search results\n                                // false by default\n    \n    objectsToSearch?:           // Which object types will be included to the search results\n        'filesOnly' |           // Only files\n        'foldersOnly' |         // Only folders\n        'all';                  // Both files and folders\n                                // all is default option\n}\n\ntype PathDetails = {\n    isFolder: boolean \n    name: string;\n    relativePath: string; // Empty for root folder\n    fullPath: string;\n    parentFolderRelativePath: string; // '..' for root folder\n    parentFolderFullPath: string;\n}\n\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rxjs from \"rxjs\";\n\nconst fs = new etl.Filesystem.Endpoint('~');\nconst scripts = ep.getFolder('scripts');\n\nconst printAllJsFileNames$ = scripts.selectRx('**/*.js').pipe(\n    rx.map(v =\u003e v.name)\n    etl.log()\n);\n\netl.run(printAllJsFileNames$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### FTP\n\nEndpoint to access files on ftp and ftps servers. Implementation based on [Basic ftp](https://www.npmjs.com/package/basic-ftp) package.\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// options: specify connection parameters\nconstructor(options: AccessOptions, verbose: boolean = false);\n\n// Creates new Collection object to get remote folder contents\n// folderPath: remote path to the ftp folder and identificator of the creating collection object\n// options: Some options how to display this endpoint\ngetFolder(folderPath: string = '.', options: CollectionOptions\u003cFileInfo\u003e = {}): Collection;\n\n// Release FilesystemCollection\n// folderPath: identificator of the releasing collection object\nreleaseFolder(folderPath: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Create the observable object and send files and folders information to it\nselectRx(): BaseObservable\u003cFileInfo\u003e;\n\n// Create folder or file. \n// remoteFolderPath, remoteFilePath, remotePath: remote path to be created\n// localFilePath: Local source file path \n// sourceStream: Source stream\n// fileContents: String as file contents\nasync insertFolder(remoteFolderPath: string);\nasync insertFile(remoteFilePath: string, localFilePath: string);\nasync insertFile(remoteFilePath: string, sourceStream: Readable);\nasync insertFileWithContents(remoteFilePath: string, fileContents: string);\n// isFolder: flag to indicate want want to add folder or file\n// Only one of localFilePath, sourceStream, contents can be specified here\nasync insert(remotePath: string, contents: { isFolder: boolean, localFilePath?: string, sourceStream?: Readable, contents?: string }); \n\n// Delete file or folder with all it's contents\n// remoteFolderPath, remoteFilePath, remotePath: Remote path to file or folder we want to delete\nasync deleteFolder(remoteFolderPath: string);\nasync deleteEmptyFolder(remoteFolderPath: string); // raise the exception if the specified folder is not empty\nasync deleteFile(remoteFilePath: string);\nasync delete(remotePath: string);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rxjs from \"rxjs\";\n\nconst ftp = new etl.filesystems.Ftp.Endpoint({host: process.env.FTP_HOST, user: process.env.FTP_USER, password: process.env.FTP_PASSWORD});\nconst folder = ftp.getFolder('/var/logs');\nconst PrintFolderContents$ = folder.selectRx().pipe(\n    etl.log()\n)\nawait etl.run(PrintFolderContents$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### SFTP\n\nEndpoint to access files by sftp. Implementation based on [ssh2-sftp-client](https://www.npmjs.com/package/ssh2-sftp-client) package.\n\n#### Endpoint\n\n#### Collection\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### WebDAV\n\nEndpoint to access remote filesystem via WebDAV protocol. Implementation based on [webdav](https://www.npmjs.com/package/webdav) package.\n\n#### Endpoint\n\n#### Collection\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Csv\n\nParses source csv file into individual records or write record to the end of destination csv file. Every record is csv-string and presented by array of values.\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// Create collection object for the specified file\n// filename: full or relative name of the csv file and identificator of the creating collection object\n// delimiter: delimiter of values in one string of file data, equals to ',' by default\n// guiOptions: Some options how to display this endpoint\ngetFile(filename: string, delimiter: string = \",\", guiOptions: CollectionGuiOptions\u003cstring[]\u003e = {}): Collection;\n\n// Release collection object\n// filename: identificator of the releasing collection object\nreleaseFile(filename: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Create the observable object and send file data to it string by string\n// skipFirstLine: skip the first line in the file, useful for skip header\n// skipEmptyLines: skip all empty lines in file\nselectRx(skipFirstLine: boolean = false, skipEmptyLines = false): Observable\u003cstring[]\u003e;\n\n// Add row to the end of file with specified value \n// value: what will be added to the file\nasync insert(value: string[]);\n\n// Clear the csv file\nasync delete();\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst csv = new etl.Csv.Endpoint('~');\nconst testFile = csv.getFile('test.csv')\n\nconst logTestFileRows$ = testFile.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logTestFileRows$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Json\n\nRead and write json file with buffering it in memory. You can get objects from json by path specifing in JSONPath format or in lodash simple path manner (see logash 'get' function documentation).\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// Create collection object for the specified file\n// filename: full or relative name of the json file and identificator of the creating collection object\n// autosave: save json from memory to the file after every change\n// autoload: load json from the file to memory before every get or search operation\n// encoding: file encoding\n// guiOptions: Some options how to display this endpoint\ngetFile(filename: string, autosave: boolean = true, autoload: boolean = false, encoding?: BufferEncoding, guiOptions: CollectionGuiOptions\u003cnumber\u003e = {}): Collection;\n\n// Release collection object\n// filename: identificator of the releasing collection object\nreleaseFile(filename: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Find and send to observable child objects by specified path\n// path: search path in lodash simple path manner\n// jsonPath: search path in JSONPath format\n// options: see below\nselectRx(path: string, options?: ReadOptions): Observable\u003cany\u003e;\nselectByJsonPath(jsonPath: string | string[], options?: ReadOptions): Observable\u003cany\u003e;\n\n// Find and return child object by specified path\n// path: search path in lodash simple path manner\n// jsonPath: search path in JSONPath format\nget(path: string): any;\ngetByJsonPath(jsonPath: string): any;\n\n// If fieldname is specified, the function find the object by path and add value as its field\n// If fieldname is not specified, the function find the array by path and push value to it\n// value: what will be added to the json\n// path: where value will be added as child, specified in lodash simple path manner\n// fieldname: name of the field to which the value will be added, \n//            and flag - is we add value to array or to object\nasync insert(value: any, path?: string, fieldname?: string);\n\n// Clear the json file and write an empty object to it\nasync delete();\n\n// Reload the json to the memory from the file\nload();\n\n// Save the json from the memory to the file\nsave();\n```\n\nTypes:\n\n```typescript\ntype JsonReadOptions = {\n    searchReturns?: 'foundedOnly'           // Default value, means that only search results objects will be sended to observable by the function\n        | 'foundedImmediateChildrenOnly'    // Only the immidiate children of search results objects will be sended to observable \n        | 'foundedWithDescendants';         // Recursive send all objects from the object tree of every search result, including search result object itself\n\n    addRelativePathAsField?: string;        // If specified, the relative path will be added to the sended objects as addRelativePathAsField field \n}\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport { tap } from \"rxjs\";\n\nconst json = new etl.Json.Endpoint('~');\nconst testFile = etl.getFile('test.json');\n\nconst printJsonBookNames$ = testFile.selectRx('store.book').pipe(\n    tap(book =\u003e console.log(book.name))\n);\n\nconst printJsonAuthors$ = testFile.selectByJsonPath('$.store.book[*].author', {searchReturns: 'foundedOnly', addRelativePathAsField: \"path\"}).pipe(\n    etl.log()\n);\n\nawait etl.run(printJsonAuthors$, printJsonBookNames$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Xml\n\n\u003ca name=\"xml\" href=\"#xml\"\u003e#\u003c/a\u003e etl.\u003cb\u003eXmlEndpoint\u003c/b\u003e(\u003ci\u003efilename, autosave?, autoload?, encoding?\u003c/i\u003e)\n\nRead and write XML document with buffering it in memory. You can get nodes from XML by path specifing in XPath format.\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// Create collection object for the specified file\n// filename: full or relative name of the xml file and identificator of the creating collection object\n// autosave: save xml from memory to the file after every change\n// autoload: load xml from the file to memory before every get or search operation\n// encoding: file encoding\n// guiOptions: Some options how to display this endpoint\ngetFile(filename: string, autosave: boolean = true, autoload: boolean = false, encoding?: BufferEncoding, guiOptions: CollectionGuiOptions\u003cstring[]\u003e = {}): Collection;\n\n// Release collection object\n// filename: identificator of the releasing collection object\nreleaseFile(filename: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Find and send to observable child objects by specified xpath\n// xpath: xpath to search\n// options: see below\nselectRx(xpath: string = '', options: XmlReadOptions = {}): EtlObservable\u003cNode\u003e;\n\n// Find and return child node by specified path\n// xpath: search path\nget(xpath: string = ''): XPath.SelectedValue\n\n// If attribute is specified, the function find the object by xpath and add value as its attribute\n// If attribute is not specified, the function find the node by xpath and push value as its child node\n// value: what will be added to the xml\n// xpath: where value will be added as child, specified in lodash simple path manner\n// attribute: name of the attribute which value will be setted, \n//            and flag - is we add value as attribute or as node\nasync insert(value: any, xpath: string = '', attribute: string = '');\n\n// Clear the xml file and write an empty object to it\nasync delete();\n\n// Reload the xml to the memory from the file\nload();\n\n// Save the xml from the memory to the file\nsave();\n```\n\nTypes:\n\n```typescript\nexport type XmlReadOptions = {\n    searchReturns?: 'foundedOnly'           // Default value, means that only search results nodes will be sended to observable by the function\n        | 'foundedImmediateChildrenOnly'    // Only the immediate children of search results nodes will be sended to observable \n        | 'foundedWithDescendants';         // Recursive send all nodes from the tree of every searched result, including searched result node itself\n\n    addRelativePathAsAttribute?: string;    // If specified, the relative path will be added to the sended nodes as attribute, specified with this value \n}\n```\n\nExample\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport { map } from \"rxjs\";\n\nconst xml = new etl.Xml.Endpoint('/tmp');\nconst testFile = xml.getFile('test.xml');\n\nconst printXmlAuthors$ = testFile.selectRx('/store/book/author').pipe(\n    map(v =\u003e v.firstChild.nodeValue),\n    etl.log()\n);\n\nawait etl.run(printXmlAuthors$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Knex\n\nRepresents common Knex database. Based on [knex engine](https://knexjs.org/).\n\n#### KnexEndpoint\n\nMethods:\n\n```typescript\nconstructor(client: ClientType, connectionString: string, pool?: PoolConfig);\nconstructor(client: ClientType, connectionConfig: ConnectionConfig, pool?: PoolConfig);\nconstructor(knexConfig: pkg.Knex.Config);\n\n// Create collection object for the specified database table\n// table: name of database table and identificator of the creating collection object\n// options: Some options how to display this endpoint\ngetTable\u003cT = Record\u003cstring, any\u003e\u003e(table: string, options: CollectionOptions\u003cstring[]\u003e = {}): KnexTableCollection\u003cT\u003e;\n\n// Create collection object for the specified sql query result\n// collectionName: identificator of the creating collection object\n// query: sql query\n// options: Some options how to display this endpoint\ngetQuery\u003cT = Record\u003cstring, any\u003e\u003e(collectionName: string, query: string, options: CollectionOptions\u003cstring[]\u003e = {}): KnexQueryCollection\u003cT\u003e;\n\n// Release collection object\n// table: identificator of the releasing collection object\nreleaseCollection(collectionName: string);\n\n// Release all collection objects, endpoint object and release connections to database.\nasync releaseEndpoint();\n```\n\n#### KnexTableCollection\n\nPresents the table from the database. \n\nMethods:\n\n```typescript\n// Create the observable object and send data from the database table to it\n// where: you can filter incoming data by this parameter\n//        it can be SQL where clause \n//        or object with fields as collumn names \n//        and its values as needed collumn values\nselectRx(where: SqlCondition\u003cT\u003e, fields?: string[]): BaseObservable\u003cT\u003e;\nselectRx(whereSql?: string, whereParams?: any[], fields?: string[]): BaseObservable\u003cT\u003e;\n\n// Insert value to the database table\n// value: what will be added to the database\nasync insert(value: T): Promise\u003cnumber[]\u003e;\nasync insert(values: T[]): Promise\u003cnumber[]\u003e;\n\n// Update all rows in database table which match to the specified condition\n// where: you can filter table rows to deleting by this parameter\n//        it can be SQL where clause \n//        or object with fields as collumn names \n//        and its values as needed collumn values\n// value: what will be set as new value for updated rows\nasync update(value: T, where: SqlCondition\u003cT\u003e): Promise\u003cnumber\u003e;\nasync update(value: T, whereSql?: string, whereParams?: any[]): Promise\u003cnumber\u003e;\n\n// Update all rows in database table which match to the specified condition\n// where: you can filter table rows to deleting by this parameter\n//        it can be SQL where clause \n//        or object with fields as collumn names \n//        and its values as needed collumn values\n// value: what will be set as new value for updated rows\nasync upsert(value: T): Promise\u003cnumber[]\u003e;\n\n// Delete rows from the database table by condition\n// where: you can filter table rows to deleting by this parameter\n//        it can be SQL where clause \n//        or object with fields as collumn names \n//        and its values as needed collumn values\nasync delete(where: SqlCondition\u003cT\u003e): Promise\u003cnumber\u003e;\nasync delete(whereSql?: string, whereParams?: any[]): Promise\u003cnumber\u003e;\n```\n\n#### KnexQueryCollection\n\nReadonly collection of sql query results. \n\nMethods:\n\n```typescript\n// Create the observable object and send data from the database table to it\n// where: you can filter incoming data by this parameter\n//        it can be SQL where clause \n//        or object with fields as collumn names \n//        and its values as needed collumn values\nselectRx(params?: any[]): BaseObservable\u003cT\u003e;\n\n// Execute query with specified parameters and return the first founded record or null\nasync get(params?: any[]): Promise\u003cT\u003e {\n\n// Execute query with specified parameters and return founded records\nasync find(params?: any[]): Promise\u003cT[]\u003e\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### CockroachDB\n\nRepresents CockroachDB database. Endpoint implementation based on KnexEndpoint.\nYou should install [node-postgres (aka 'pg') package](https://github.com/brianc/node-postgres) module to use this endpoint! \n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.CockroachDb.Endpoint('postgres://user:password@127.0.0.1:5432/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### MariaDB\n\nRepresents MariaDB database. Endpoint implementation based on KnexEndpoint.\nYou should install [mysql package](https://www.npmjs.com/package/mysql) module to use this endpoint! \n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig, driver?: 'mysql' | 'mysql2');\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig, driver?: 'mysql' | 'mysql2');\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.MariaDb.Endpoint('mysql://user:password@127.0.0.1:3306/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### MS SQL Server\n\nRepresents MS SQL Server database. Endpoint implementation based on KnexEndpoint.\nYou should install [tedious package](https://github.com/tediousjs/tedious) module to use this endpoint! \n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.SqlServer.Endpoint('mssql://user:password@127.0.0.1:1433/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### MySQL\n\nRepresents MySQL database. Endpoint implementation based on KnexEndpoint.\nYou should install [mysql package](https://www.npmjs.com/package/mysql) module to use this endpoint! \n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig, driver?: 'mysql' | 'mysql2');\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig, driver?: 'mysql' | 'mysql2');\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.MySql.Endpoint('mysql://user:password@127.0.0.1:3306/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Oracle DB\n\nRepresents Oracle database. Endpoint implementation based on KnexEndpoint.\nYou should install [oracledb package](https://www.npmjs.com/package/oracledb) module to use this endpoint! \n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.OracleDb.Endpoint({\n    host: config.oracle.host,\n    user: config.oracle.user,\n    password: config.oracle.password,\n    database: config.oracle.database,\n});\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Postgres\n\nRepresents PostgreSQL database. Endpoint implementation based on KnexEndpoint.\nYou should install [node-postgres (aka 'pg') package](https://github.com/brianc/node-postgres) module to use this endpoint!\n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.Postgres.Endpoint('postgres://user:password@127.0.0.1:5432/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Amazone Redshift\n\nRepresents Amazone Redshift database. Endpoint implementation based on KnexEndpoint.\nYou should install [node-postgres (aka 'pg') package](https://github.com/brianc/node-postgres) module to use this endpoint!\n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.Redshift.Endpoint('postgres://user:password@127.0.0.1:5439/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### SQLite\n\nRepresents SQLite3 database. Endpoint implementation based on KnexEndpoint.\nYou should install [sqlite3 package](https://www.npmjs.com/package/sqlite3) module to use this endpoint!\n\n#### Endpoint\n\nExtends KnexEndpoint and contains all it's methods.\n\nConstructors:\n\n```typescript\nconstructor(connectionString: string, pool?: PoolConfig);\nconstructor(connectionConfig: ConnectionConfig, pool?: PoolConfig);\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.databases.SqlLite.Endpoint(connection: {\n    filename: \"./mydb.sqlite\"\n});\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.log()\n);\n\netl.run(logUsers$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### GeneralEmail\n\nEndpoint to send and recive emails. Implements SMTP protocol and based on [nodemailer](https://nodemailer.com/) library.\n\n#### Endpoint\n\nMethods:\n\n```typescript\nconstructor(connectionOptions: ConnectionOptions);\nasync releaseEndpoint();\n\nasync send(to: string[] | string, subject: string, body: string, cc?: string[] | string, bcc?: string[] | string, from?: string): Promise\u003cSendError | undefined\u003e;\nasync send(value: EMail): Promise\u003cSendError | undefined\u003e;\n\ngetInbox(options: CollectionOptions\u003cEMail\u003e = {}): Collection;\nreleaseInbox();\ngetMailbox(mailBox: string, options: CollectionOptions\u003cEMail\u003e = {}): Collection;\nreleaseMailbox(mailBox: string);\n```\n\n#### Collection\n\nPresents emails mailbox. \n\nMethods:\n\n```typescript\n// Create the observable object and send product data from the Magento to it\nselectRx(): BaseObservable\u003cEMail\u003e;\nselectRx(searchOptions: SearchOptions, markSeen?: boolean): BaseObservable\u003cEMail\u003e;\nselectRx(range: string, markSeen?: boolean): BaseObservable\u003cEMail\u003e;\nselectRx(searchCriteria: any[], markSeen?: boolean ): BaseObservable\u003cEMail\u003e;\n\nasync get(UID: string | number, markSeen: boolean = false): Promise\u003cEMail\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst gmail = new etl.messangers.Gmail.Endpoint(process.env.GMAIL_USER!, process.env.GMAIL_PASSWORD!);\nconst inbox = gmail.getInbox();\n\nconst PrintMails$ = inbox.selectRx({seen: false}).pipe(\n    rx.take(10),\n    etl.log()\n)\n\nawait etl.run(PrintMails$);\n\nconst mail = await inbox.get(1463);\nconsole.log(mail);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Gmail\n\nEndpoint to work with Gmail service. Based on [GeneralEmail](#generalemail) endpoint.\n\nYou should to get the application password from Gmail service to use this endpoint. \nFollow this steps to get it:\n\n1. Login to you Gmail account\n2. Open this link https://myaccount.google.com/security\n3. Enable 2 factor authentication\n4. Go to https://myaccount.google.com/apppasswords\n5. From Select App options select Other and write your app name (it could be any name like mycustomapp)\n6. It will generate you the password - copy the password from the popup\n7. Use that copied password in the application password parameter in the Gmail endpoint constructor.\n\n[![Gmail security](/static/gmail-security.png)](/static/gmail-security.png)\n\n#### Endpoint\n\nMethods:\n\n```typescript\nconstructor(userEmail: string, appPassword: string);\nasync releaseEndpoint();\n\nasync send(to: string[] | string, subject: string, body: string, cc?: string[] | string, bcc?: string[] | string, from?: string): Promise\u003cSendError | undefined\u003e;\nasync send(value: EMail): Promise\u003cSendError | undefined\u003e;\n\ngetInbox(options: CollectionOptions\u003cEMail\u003e = {}): Collection;\nreleaseInbox();\ngetMailbox(mailBox: string, options: CollectionOptions\u003cEMail\u003e = {}): Collection;\nreleaseMailbox(mailBox: string);\n```\n\n#### Collection\n\nPresents emails mailbox. \n\nMethods:\n\n```typescript\n// Create the observable object and send product data from the Magento to it\nselectRx(): BaseObservable\u003cEMail\u003e;\nselectRx(searchOptions: SearchOptions, markSeen?: boolean): BaseObservable\u003cEMail\u003e;\nselectRx(range: string, markSeen?: boolean): BaseObservable\u003cEMail\u003e;\nselectRx(searchCriteria: any[], markSeen?: boolean ): BaseObservable\u003cEMail\u003e;\n\nasync get(UID: string | number, markSeen: boolean = false): Promise\u003cEMail\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst gmail = new etl.messangers.Gmail.Endpoint(process.env.GMAIL_USER!, process.env.GMAIL_PASSWORD!);\nconst inbox = gmail.getInbox();\n\nconst PrintMails$ = inbox.selectRx({seen: false}).pipe(\n    rx.take(10),\n    etl.log()\n)\n\nawait etl.run(PrintMails$);\n\nconst mail = await inbox.get(1463);\nconsole.log(mail);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### SMS.RU\n\nEndpoint to work with http://sms.ru service. \n\nYou should have the account on this service to use it. \n\n#### Endpoint\n\nMethods:\n\n```typescript\nconstructor(apiId: string);\n\nasync send(message: string, toPhone: string, from?: string): Promise\u003cSendError | undefined\u003e;\nasync send(message: string, toPhones: string[], from?: string): Promise\u003cSendError | undefined\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst sms = new etl.messangers.SmsRu();\nconst err = await sms.sendSms('hello', '123-45-67');\nif (err) console.log(err); // log error if any\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Magento\n\nPresents Magento CMS objects via [Magento REST API](https://devdocs.magento.com/guides/v2.4/rest/tutorials/index.html).\n\nIt uses token-based authentication to access API. You must register a web service on Admin. Use the following general steps to set up Magento to enable web services:\n\n1. Create a web services user on Admin by selecting System \u003e Permission \u003e All Users \u003e Add New User. (If you are using session-based or OAuth authentication, you do not need to create the new user in the Admin.)\n2. Create a new integration on Admin. To create an integration, click System \u003e Extensions \u003e Integration \u003e Add New Integration. Be sure to restrict which resources the integration can access.\n\nSee the [official guide](https://devdocs.magento.com/guides/v2.3/get-started/bk-get-started-api.html) for more information. Or you can go to [this article ](https://meetanshi.com/blog/create-update-product-using-rest-api-in-magento-2/) for details how to configure Magento integration to get access to it's API. \n\n#### Endpoint\n\nMethods:\n\n```typescript\n// magentoUrl: Url of Magento\n// login: admin login\n// password: admin password\n// rejectUnauthorized: You can set it to true to ignore ssl servificate problems while development.\nconstructor(magentoUrl: string, login: string, password: string, rejectUnauthorized: boolean = true);\n\n// Create collection object for the Magento products\n// guiOptions: Some options how to display this endpoint\ngetProducts(guiOptions: CollectionGuiOptions\u003cPartial\u003cProduct\u003e\u003e = {}): ProductsCollection;\n\n// Release products collection object\nreleaseProducts();\n```\n\n#### ProductsCollection\n\nPresents Magento CMS products. \n\nMethods:\n\n```typescript\n// Create the observable object and send product data from the Magento to it\n// where: you can filter products by specifing object with fields as collumn names and it's values as fields values \n// fields: you can select which products fields will be returned (null means 'all fields') \nselectRx(where: Partial\u003cProduct\u003e = {}, fields: (keyof Product)[] = null): BaseObservable\u003cPartial\u003cProduct\u003e\u003e ;\n\n// Add new product to the Magento\n// value: product fields values\nasync insert(value: NewProductAttributes);\n\n// Upload image to the magento and set it as image of specified product and returns total count of images for this product\n// product: product sku\n// imageContents: binary form of the image file\n// filename: name of the file in with magento will store the image\n// label: label of the product image\n// type: mime type of the image\nasync uploadImage(product: {sku: string} | string, imageContents: Blob, filename: string, label: string, type: \"image/png\" | \"image/jpeg\" | string): Promise\u003cnumber\u003e;\n// Operator to upload product image from the pipe\nuploadImageOperator\u003cT\u003e(func: (value: T) =\u003e {product: {sku: string} | string, imageContents: Blob, filename: string, label: string, type: \"image/png\" | \"image/jpeg\" | string}): OperatorFunction\u003cT, T\u003e;\n\n// Utility static function to get products as array\nstatic async getProducts(endpoint: Endpoint, where: Partial\u003cProduct\u003e = {}, fields: (keyof Product)[] = null): Promise\u003cPartial\u003cProduct\u003e[]\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst magento = new etl.Magento.Endpoint('https://magento.test', process.env.MAGENTO_LOGIN!, process.env.MAGENTO_PASSWORD!);\nconst products = magento.getProducts();\n\nconst logProductsWithPrice100$ = products.selectRx({price: 100}).pipe(\n    etl.log()\n);\n\netl.run(logProductsWithPrice100$)\n```\n\n#### StockCollection\n\nPresents Magento CMS stock items. Stock items - is products on stock.\n\nMethods:\n\n```typescript\n// Create the observable object and send stock items data from the Magento to it\n// sku, product: you can filter stock items by product attributes\nselectRx(sku: string): BaseObservable\u003cStockItem\u003e;\nselectRx(product: Partial\u003cProduct\u003e): BaseObservable\u003cStockItem\u003e;\n\n// Get stock item for specified product\n// sku, product: product, wich stock items we need to get\npublic async getStockItem(sku: string): Promise\u003cStockItem\u003e;\npublic async getStockItem(product: {sku: string}): Promise\u003cStockItem\u003e;\n\n// Update product stock quantity \npublic async updateStockQuantity(sku: string, quantity: number);\npublic async updateStockQuantity(product: {sku: string}, quantity: number);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Trello\n\nPresents Trello task tracking system objects.\nFor details how to get API key and authorization token please read [Trello documentation](https://developer.atlassian.com/cloud/trello/guides/rest-api/api-introduction/).\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// url: Trello web url\n// apiKey: Trello API key\n// authToken: Trello authorization token\n// rejectUnauthorized: You can set it to true to ignore ssl servificate problems while development.\nconstructor(apiKey: string, authToken: string, url: string = \"https://trello.com\", rejectUnauthorized: boolean = true);\n\n// Create collection object for the Trello user boards\n// username: user, which boards we need to get, by default it is a Trello authorization token owner\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetUserBoards(username: string = 'me', collectionName: string = 'Boards', guiOptions: CollectionGuiOptions\u003cPartial\u003cBoard\u003e\u003e = {}): BoardsCollection;\n\n// Create collection object for the Trello board lists\n// boardId: board id\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetBoardLists(boardId: string, collectionName: string = 'Lists', guiOptions: CollectionGuiOptions\u003cPartial\u003cList\u003e\u003e = {}): ListsCollection;\n\n// Create collection object for the Trello list cards\n// listId: list id\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetListCards(listId: string, collectionName: string = 'Cards', guiOptions: CollectionGuiOptions\u003cPartial\u003cCard\u003e\u003e = {}): CardsCollection;\n\n// Create collection object for the Trello card comments\n// cardId: card id\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetCardComments(cardId: string, collectionName: string = 'Comments', guiOptions: CollectionGuiOptions\u003cPartial\u003cComment\u003e\u003e = {}): CommentsCollection;\n\n// Release collection data\n// collectionName: identificator of the releasing collection object\nreleaseCollection(collectionName: string);\n```\n\n#### BoardsCollection\n\nPresents Trello boards accessible by user which was specified while collection creation. \n\nMethods:\n\n```typescript\n// Create the observable object and send boards data from the Trello to it\n// where (does not working now!): you can filter boards by specifing object with fields as collumn names and it's values as fields values \n// fields (does not working now!): you can select which board fields will be returned (null means 'all fields') \nselectRx(where: Partial\u003cBoard\u003e = {}, fields: (keyof Board)[] = null): EtlObservable\u003cPartial\u003cBoard\u003e\u003e;\n\n// Add new board to the Trello\n// value: board fields values\nasync insert(value: Omit\u003cPartial\u003cBoard\u003e, 'id'\u003e);\n\n// Update board fields values by board id\n// boardId: board id\n// value: new board fields values as hash object\nasync update(boardId: string, value: Omit\u003cPartial\u003cBoard\u003e, 'id'\u003e);\n\n// Get all user boards\nasync get(): Promise\u003cBoard[]\u003e;\n\n// Get board by id\n// boardId: board id\nasync get(boardId?: string): Promise\u003cBoard\u003e;\n\n// Get board by url from browser\nasync getByBrowserUrl(url: string): Promise\u003cBoard\u003e;\n```\n\n#### ListsCollection\n\nPresents Trello lists on board which was specified while collection creation. \n\nMethods:\n\n```typescript\n// Create the observable object and send lists data from the Trello to it\n// where (does not working now!): you can filter lists by specifing object with fields as collumn names and it's values as fields values \n// fields (does not working now!): you can select which list fields will be returned (null means 'all fields') \nselectRx(where: Partial\u003cList\u003e = {}, fields: (keyof List)[] = null): EtlObservable\u003cPartial\u003cList\u003e\u003e;\n\n// Add new list to the Trello\n// value: list fields values\nasync insert(value: Omit\u003cPartial\u003cList\u003e, 'id'\u003e);\n\n// Update list fields values by list id\n// listId: list id\n// value: new list fields values as hash object\nasync update(listId: string, value: Omit\u003cPartial\u003cList\u003e, 'id'\u003e);\n\n// Get all lists\nasync get(): Promise\u003cList[]\u003e;\n\n// Get list by id\n// listId: list id\nasync get(listId?: string): Promise\u003cList\u003e;\n\n// Archive or unarchive a list\n// listId: list id\nasync switchClosed(listId: string);\n\n// Move list to another board\n// listId: list id\n// destBoardId: destination board id\nasync move(listId: string, destBoardId: string);\n\n// Get list actions\n// listId: list id\nasync getActions(listId: string);\n```\n\n#### CardsCollection\n\nPresents Trello cards in list which was specified while collection creation. \n\nMethods:\n\n```typescript\n// Create the observable object and send cards data from the Trello to it\n// where (does not working now!): you can filter cards by specifing object with fields as collumn names and it's values as fields values \n// fields (does not working now!): you can select which card fields will be returned (null means 'all fields') \nselectRx(where: Partial\u003cCard\u003e = {}, fields: (keyof Card)[] = null): EtlObservable\u003cPartial\u003cCard\u003e\u003e;\n\n// Add new card to the Trello\n// value: card fields values\nasync insert(value: Omit\u003cPartial\u003cCard\u003e, 'id'\u003e);\n\n// Update card fields values by card id\n// listId: card id\n// value: new list fields values as hash object\nasync update(cardId: string, value: Omit\u003cPartial\u003cCard\u003e, 'id'\u003e);\n\n// Get all cards\nasync get(): Promise\u003cCard[]\u003e;\n\n// Get card by id\n// cardId: card id\nasync get(cardId?: string): Promise\u003cCard\u003e;\n\n// Archive all cards in current list\nasync archiveListCards();\n\n// Move all cards from the current list to another board and list\n// destBoardId: destination board id\n// destListId: destination list id\nasync moveListCards(destBoardId: string, destListId: string);\n```\n\n#### CommentsCollection\n\nPresents Trello card comments in card which was specified while collection creation. \n\nMethods:\n\n```typescript\n// Create the observable object and send comments from the Trello to it\n// where (does not working now!): you can filter comments by specifing object with fields as collumn names and it's values as fields values \n// fields (does not working now!): you can select which comment fields will be returned (null means 'all fields') \nselectRx(where: Partial\u003cComment\u003e = {}, fields: (keyof Comment)[] = null): EtlObservable\u003cPartial\u003cComment\u003e\u003e;\n\n// Add new comment to the Trello card\n// text: comment text\nasync insert(text: string);\n\n// Update comment fields values by comment id\n// commentId: comment id\n// value: new comment fields values as hash object\nasync update(commentId: string, value: Omit\u003cPartial\u003cComment\u003e, 'id'\u003e);\n\n// Get all comments\nasync get(): Promise\u003cComment[]\u003e;\n\n// Get comment by id\n// commentId: card id\nasync get(commentId?: string): Promise\u003cComment\u003e;\n```\n\nExample:\n\n```typescript\nimport * as rx from 'rxjs';\nimport * as etl from 'etl-gun';\n\nconst trello = new etl.Trello.Endpoint(process.env.TRELLO_API_KEY!, process.env.TRELLO_AUTH_TOKEN!);\n\nconst boards = trello.getUserBoards();\nconst board = await boards.getByBrowserUrl('https://trello.com/b/C9zegsyz/board1');\n\nconst lists = trello.getBoardLists(board.id);\nconst list = (await lists.get())[0];\n\nconst cards = trello.getListCards(list.id);\n\nconst addCommentToAllCards$ = cards.selectRx().pipe(\n    rx.tap(card =\u003e {\n        const comments = trello.getBoardLists(card.id, 'cards');\n        comments.push('New comment');\n        trello.releaseCollection('cards');\n    })\n);\n\netl.run(addCommentToAllCards$)\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Zendesk\n\nPresents Zendesk task tracking system objects.\nFor details how to get API key and authorization token please read [Zendesk documentation](https://developer.zendesk.com/api-reference/).\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// zendeskUrl: Zendesk web url\n// username: user login\n// token: Zendesk authorization token\n// rejectUnauthorized: You can set it to true to ignore ssl servificate problems while development.\nconstructor(zendeskUrl: string, username: string, token: string, rejectUnauthorized: boolean = true);\n\n// Create collection object for the Zendesk tickets\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetTickets(collectionName: string = 'Tickets', options: CollectionOptions\u003cPartial\u003cTicket\u003e\u003e = {}): TicketsCollection;\n\n// Create collection object for the Zendesk tickets fields\n// collectionName: identificator of the creating collection object\n// guiOptions: Some options how to display this endpoint\ngetTicketFields(collectionName: string = 'TicketFields', options: CollectionOptions\u003cPartial\u003cField\u003e\u003e = {}): TicketFieldsCollection;\n\n// Release collection data\n// collectionName: identificator of the releasing collection object\nreleaseCollection(collectionName: string);\n```\n\n#### TicketsCollection\n\nPresents all Zendesk tickets. \n\nMethods:\n\n```typescript\n// Create the observable object and send tickets data from the Zendesk to it\n// where: you can filter tickets by specifing object with fields as collumn names and it's values as fields values \nselectRx(where: Partial\u003cTicket\u003e = {}): BaseObservable\u003cPartial\u003cTicket\u003e\u003e;\n\n// Add new ticket to the Zendesk\n// value: ticket fields values\nasync insert(value: Omit\u003cPartial\u003cTicket\u003e, 'id'\u003e);\n\n// Update ticket fields values by ticket id\n// ticketId: ticket id\n// value: new ticket fields values as hash object\nasync update(ticketId: number, value: Omit\u003cPartial\u003cTicket\u003e, 'id'\u003e);\n\n// Get all tickets\nasync get(): Promise\u003cTicket[]\u003e;\n\n// Get ticket by id\n// ticketId: ticket id\nasync get(ticketId: number): Promise\u003cTicket\u003e;\n```\n\n#### TicketFieldsCollection\n\nPresents all Zendesk tickets fields. \n\nMethods:\n\n```typescript\n// Create the observable object and send fields description data from the Zendesk to it\nselectRx(): BaseObservable\u003cPartial\u003cField\u003e\u003e;\n\n// Add new field to the Zendesk\n// value: field attributes values\nasync insert(value: Omit\u003cPartial\u003cField\u003e, 'id'\u003e);\n\n// Update field attributes by field id\n// fieldId: field id\n// value: new field attributes values as hash object\nasync update(fieldId: number, value: Omit\u003cPartial\u003cField\u003e, 'id'\u003e);\n\n// Get all fields\nasync get(): Promise\u003cField[]\u003e;\n\n// Get field by id\n// fieldId: field id\nasync get(fieldId: number): Promise\u003cField\u003e;\n```\n\nExample:\n\n```typescript\nimport * as rx from 'rxjs';\nimport * as etl from 'etl-gun';\n\nconst zendesk = new etl.Zendesk.Endpoint(process.env.ZENDESK_URL!, process.env.ZENDESK_USERNAME!, process.env.ZENDESK_TOKEN!);\nconst tickets = zendesk.getTickets();\n\nconst PrintAllOpenedTickets$ = tickets.selectRx().pipe(\n    etl.where({status: 'open'}),\n    etl.log()\n)\n\netl.run(PrintAllOpenedTickets$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Telegram\n\nWith this endpoint you can create telegram bots and chats with users. It can listen for user messages and send the response massages. \nIt also can set the user keyboard for the chat.\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// Start bot and return collection object for the bot messages\n// collectionName: identificator of the creating collection object\n// token: Bot token\n// keyboard: JSON keyboard description, see the node-telegram-bot-api for detailes\n//           Keyboard example: [[\"Text for command 1\", \"Text for command 2\"], [\"Text for command 3\"]]\n// guiOptions: Some options how to display this endpoint\nstartBot(collectionName: string, token: string, keyboard?: any, guiOptions: CollectionGuiOptions\u003cTelegramInputMessage\u003e = {}): Collection;\n\n// Stop bot\n// collectionName: identificator of the releasing collection object\nreleaseBot(collectionName: string);\n```\n\n#### Collection\n\nPresents all chat bot messages.\n\nMethods:\n\n```typescript\n// Start reciving of all users messages\nselectRx(): Observable\u003cT\u003e;\n\n// Stop reciving of user messages\nasync stop();\n\n// Pushes message to the chat\n// value: Message in TelegramInputMessage type\n// chatId: id of the destination chat, get it from input user messages\n// message: Message to send\nasync insert(value: TelegramInputMessage);\nasync insert(chatId: string, message: string);\n\n// Update keyboard structure to specified\n// keyboard: JSON keyboard description, constructor for detailes\nsetKeyboard(keyboard: any)\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst telegram = new etl.Telegram.Endpoint();\nconst bot = telegram.startBot('bot 1', '**********');\n\nconst startTelegramBot$ = bot.selectRx().pipe(\n    etl.log(),          // log user messages to the console\n    etl.push(bot)  // echo input message back to the user\n);\n\netl.run(startTelegramBot$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n### Interval\n\nThis endpoint is analog of **RxJs** interval() operator, with GUI support. It emits simple counter, which increments every interval.\n\n#### Endpoint\n\nMethods:\n\n```typescript\n// Create new interval collection object\n// collectionName: identificator of the creating collection object\n// interval: Time interval in milliseconds between two emitted values \n// guiOptions: Some options how to display this endpoint\ngetSequence(collectionName: string, interval: number, guiOptions: CollectionGuiOptions\u003cnumber\u003e = {}): Collection;\n\n// Stop interval\n// collectionName: identificator of the releasing collection object\nreleaseSequence(collectionName: string);\n```\n\n#### Collection\n\nMethods:\n\n```typescript\n// Start interval generation, create observable and emit counter of intervals to it\nselectRx(): Observable\u003cnumber\u003e;\n\n// Stop endpoint reading\nasync stop();\n\n// Set value of interval counter\n// value: new value of the interval counter\nasync insert(value: number);\n\n// Set interval counter to 0\nasync delete();\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst timer = new etl.Interval.Endpoint();\nconst seq = new etl.getSequence('every 500 ms', 500);\n\nconst startTimer$ = seq.selectRx().pipe(\n    etl.log()          // log counter\n);\n\netl.run(startTimer$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n## Operators\n\nApart from operators from this library, you can use any operators of **RxJs** library.\n\n### run\n\nThis function runs one or several streams and return promise to waiting when all streams are complites.\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet memory = new etl.Memory.Endpoint();\nlet buffer = memory.getBuffer('test buffer', [1, 2, 3, 4, 5]);\n\nlet stream$ = buffer.selectRx().pipe(\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### log\n\n\u003ca name=\"log\" href=\"#log\"\u003e#\u003c/a\u003e etl.\u003cb\u003elog\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nPrints the value from the stream to the console.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rx from \"rxjs\";\n\nlet stream$ = rx.interval(1000).pipe(\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### expect\n\nThis function checks condition end if false - throws an error to the error collection.\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet memory = new etl.Memory.Endpoint();\nlet buffer = memory.getBuffer('test buffer', [{count: 1}, {count: 2}]);\n\nconst errorsEndpoint = etl.Errors.getEndpoint();\nconst errors = errorsEndpoint.getCollection('all');\n\nlet stream$ = buffer.selectRx().pipe(\n    etl.expect('count = 1', { count: 1 }, errors),\n    etl.expect('count one of [1,2,3]', { count: etl.VALUE.of([1,2,3]) }),\n    etl.expect('count not \u003e 3', { count: etl.VALUE.not['\u003e'](3) }),\n    etl.expect('count check function', { count: v =\u003e v \u003c 5 }),\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### where\n\nVariants:\n\n```typescript\nfunction where\u003cT\u003e(condition: Condition\u003cT\u003e): OperatorFunction\u003cT, T\u003e;\n```\n\nThis operator is analog of **where** operation in SQL and is synonym of the **filter** operator from the **RxJS** library - but with improvements. It cat skip some values from the input stream by the specified condition. You can specify predicate function to determine filter conditions or you can specify map object as condition (like typeorm 'where' parameter in find() method).\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rx from \"rxjs\";\n\nlet stream$ = rx.interval(1000).pipe(\n    etl.where(v =\u003e v % 2 === 0),\n    etl.where('count = 1', { count: 1 }),\n    etl.where('count one of [1,2,3]', { count: etl.VALUE.of([1,2,3]) }),\n    etl.where('count not \u003e 3', { count: etl.VALUE.not['\u003e'](3) }),\n    etl.log()\n);\netl.run(stream$);\n```\n\n### collect\n\nVariants:\n\n```typescript\nfunction collect\u003cT\u003e(maxLength: number): rx.OperatorFunction\u003cT, T[]\u003e;\nfunction collect\u003cT\u003e(startNewBuf: (value: T, buffer: T[]) =\u003e boolean): rx.OperatorFunction\u003cT, T[]\u003e;\n```\n\nThis operator is analog of **buffer** operators from the **RxJS** library - but with improvements. It cat start new buffer by condition, or by buffer length.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rx from \"rxjs\";\n\nconst src = rx.of(1,2,3,4,5,6,7,8,9,10);\nconst p$ = src.pipe(\n    etl.collect(v =\u003e v % 3 == 1 ),\n    etl.log()\n)\netl.run(p$);\n```\n\n### push\n\n\u003ca name=\"push\" href=\"#push\"\u003e#\u003c/a\u003e etl.\u003cb\u003epush\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator call the **Endpoint.push** method to push value from stream to the specified endpoint.\n\nVariants:\n\n```typescript\n// Push value to collection with no wait for result\nfunction push\u003cS, T=S\u003e(collection: BaseCollection\u003cT\u003e, options?: PushOptions\u003cS, T\u003e | null): OperatorFunction\u003cS, S\u003e;\n\n// Push value to collection with waiting for result\nfunction pushAndWait\u003cS, T=S\u003e(collection: BaseCollection\u003cT\u003e, options?: PushOptions\u003cS, T\u003e | null): OperatorFunction\u003cS, S\u003e;\n\n// Push value to collection, wait for result, send result to log\nfunction pushAndLog\u003cS, T=S\u003e(collection: BaseCollection\u003cT\u003e, options?: PushOptions\u003cS, T\u003e | null): OperatorFunction\u003cS, S\u003e;\n\n// Push value to collection, get the result and put it as stream value or as property of stream value\nfunction pushAndGet\u003cS, T, R\u003e(collection: BaseCollection\u003cT\u003e, options?: PushOptions\u003cS, T\u003e \u0026 {toProperty: string} | null): OperatorFunction\u003cS, R\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rx from \"rxjs\";\n\nlet csv = new etl.Csv.Endpoint();\nlet dest = csv.getFile('test.csv');\n\nlet stream$ = rx.interval(1000).pipe(\n    etl.push(dest)\n);\n\netl.run(stream$);\n```\n\n### rools\n\n\u003ca name=\"push\" href=\"#push\"\u003e#\u003c/a\u003e etl.\u003cb\u003erools\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator integrates etl engine with [Rools](https://www.npmjs.com/package/rools) business rules engine. It calls the specified rule set and put the current stream data value to it as a fact, and then uses the result of rules execution as a new stream value.\n\nIt allows separate business logic of data analysis and transformation from other etl code. You can load rules from any source at runtime and allow to modify it by the end users. \n\nIn rules you can:\n* Analyse and change any stream value properties\n* Call any async methods from 'then' section (the 'when' section is sync, it is a rule engine specific, but 'then' is fully async compatible)\n* Add 'etl' property to value with some control flow instructions for etl engine:\n  * value.etl.skip - set to true to skip futher processing of this value\n  * value.etl.stop - set to true to stop processing of all remaining collection values\n  * value.etl.error - set to error message if any error was founded (it raise the exception and stop futher processing)\n* Specify priority of rules\n* Use rules inheritance\n\nYou can write you rules in any order, it does not metter to rules engine. See [rools documentation](https://www.npmjs.com/package/rools) for details.\n\nSpecification:\n\n```typescript\ntype EtlRoolsResult = {\n    etl?: {\n        skip?: boolean;\n        stop?: boolean;\n        error?: string;\n    }\n}\n\nfunction rools\u003cT, R = T\u003e(rools: Rools): rx.OperatorFunction\u003cT, R\u003e;\n```\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\nimport * as rx from \"rxjs\";\nimport { Rools, Rule } from 'rools';\n\ntype DbProduct = {\n    id: number, \n    name: string, \n    price: number, \n    tax_class_id?: number\n}\n\nconst ruleSkipCheapProducts = new Rule({\n    name: 'skip products with price \u003c= 1000',\n    when: (product: DbProduct) =\u003e product.price! \u003c= 1000,\n    then: (product: DbProduct \u0026 EtlRoolsResult) =\u003e {\n        product.etl = {skip: true};\n    },\n});\n\nconst ruleSetProductTaxClass = new Rule({\n    name: 'update product tax class',\n    when: (product: DbProduct) =\u003e product.price! \u003e 1000,\n    then: (product: DbProduct \u0026 EtlRoolsResult) =\u003e {\n        product.tax_class_id = 10;\n    },\n});\n\nconst rules = new Rools();\nawait rules.register([ruleSkipCheapProducts, ruleSetProductTaxClass]);\n\nconst db = new etl.databases.MySql.Endpoint(process.env.MYSQL_CONNECTION_STRING!, undefined, 'mysql2');\nconst table = db.getTable\u003cDbProduct\u003e('products');\n\nconst PrintRichProducts$ = table.selectRx().pipe(\n    etl.rools(rules),\n    etl.log()\n)\n\nawait etl.run(PrintRichProducts$);\n```\n\n### move\n\n\u003ca name=\"numerate\" href=\"#numerate\"\u003e#\u003c/a\u003e etl.\u003cb\u003etoProperty\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator moves to specified property the whole stream value or it's property. Lodash paths is supported.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst memory = etl.Memory.getEndpoint();\nconst buf = memory.getBuffer\u003cnumber\u003e('buf', [1,2,3,4,5]);\n\nlet stream$ = src.selectRx().pipe(\n    etl.move\u003c{ nn: number }\u003e({to: 'nn'}), // 1 -\u003e { nn: 1 }\n    etl.move\u003c{ num: number }\u003e({from: 'nn', to: 'num'}), // { nn: 1 } -\u003e { num: 1 }\n    etl.copy\u003c{ num: number, kk: {pp: number} }\u003e('nn', 'kk.pp'), // { nn: 1 } -\u003e { nn: 1, kk: {pp: 1} }\n\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### copy\n\n\u003ca name=\"numerate\" href=\"#numerate\"\u003e#\u003c/a\u003e etl.\u003cb\u003etoProperty\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator copy the specified property of the stream value to the another property. Lodash paths is supported.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst memory = etl.Memory.getEndpoint();\nconst buf = memory.getBuffer\u003cnumber\u003e('buf', [1,2,3,4,5]);\n\nlet stream$ = src.selectRx().pipe(\n    etl.move\u003c{ nn: number }\u003e({to: 'nn'}), // 1 -\u003e { nn: 1 }\n    etl.move\u003c{ num: number }\u003e({from: 'nn', to: 'num'}), // { nn: 1 } -\u003e { num: 1 }\n    etl.copy\u003c{ num: number, kk: {pp: number} }\u003e('nn', 'kk.pp'), // { nn: 1 } -\u003e { nn: 1, kk: {pp: 1} }\n\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### numerate\n\n\u003ca name=\"numerate\" href=\"#numerate\"\u003e#\u003c/a\u003e etl.\u003cb\u003enumerate\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator enumerate input values and add index field to value if it is object or index column if value is array. If the input stream values is objects, you should specify index field name as the second parameter of operator.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet csv = new etl.Csv.Endpoint();\nlet src = csv.getFile('test.csv');\n\nlet stream$ = src.selectRx().pipe(\n    etl.numerate(10), // 10 is the first value for numeration\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### addField\n\n\u003ca name=\"numerate\" href=\"#numerate\"\u003e#\u003c/a\u003e etl.\u003cb\u003eaddField\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator applicable to the stream of objects. It calculate callback function and add result as new field to the input stream value.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nconst pg = new etl.Postgres.Endpoint('postgres://user:password@127.0.0.1:5432/database');\nconst table = pg.getTable('users');\n\nconst logUsers$ = table.selectRx().pipe(\n    etl.addField('NAME_IN_UPPERCASE', value =\u003e value.name.toUpperCase()),\n    etl.log()\n);\n\netl.run(logUsers$);\n```\n\n### addColumn\n\n\u003ca name=\"numerate\" href=\"#numerate\"\u003e#\u003c/a\u003e etl.\u003cb\u003eaddColumn\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator applicable to the stream of arrays. It calculate callback function and add result as a new column to the input stream value.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet csv = new etl.Csv.Endpoint();\nlet src = csv.getFile('test.csv');\n\nconst stream$ = src.selectRx().pipe(\n    etl.addColumn(value =\u003e value[2].toUpperCase()), \n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### join\n\n\u003ca name=\"join\" href=\"#join\"\u003e#\u003c/a\u003e etl.\u003cb\u003ejoin\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator is analog of join operation in SQL. It takes the second input stream as the parameter, and gets all values from this second input stream for every value from the main input stream. Then it merges both values to one object (if values are objects) or to one array (if at least one of values are array), and put the result value to the main stream.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet csv = new etl.Csv.Endpoint();\nlet src = csv.getFile('test.csv');\n\nlet mem = new etl.Memory.Endpoint();\nlet buffer = mem.getBuffer('buffer 1', [1, 2, 3, 4, 5]);\n\nlet stream$ = src.selectRx().pipe(\n    etl.join(buffer),\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n### mapAsync\n\n\u003ca name=\"mapAsync\" href=\"#mapAsync\"\u003e#\u003c/a\u003e etl.\u003cb\u003emapAsync\u003c/b\u003e([\u003ci\u003eoptions\u003c/i\u003e])\n\nThis operator is analog of rxjs map operator for async callback function. It call and wait for callback and then use it's result as new stream item.\n\nExample:\n\n```typescript\nimport * as etl from \"etl-gun\";\n\nlet mem = new etl.Memory.Endpoint();\nlet buffer = mem.getBuffer('urls', ['1.json', '2.json', '3.json']);\n\nconst mySite = new HttpClientHelper('http://www.mysite.com/jsons');\n\nlet stream$ = src.selectRx().pipe(\n    mapAsync(async (url) =\u003e await mySite.getJson(url)),\n    etl.log()\n);\n\netl.run(stream$);\n```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n## Misc\n\n### GoogleTranslateHelper\n\nThis class help you to use Google translate service.\n\n```typescript\nimport { Csv, GoogleTranslateHelper, log, run } from \"etl-gun\";\n\nlet csv = new Csv.Endpoint();\nlet src = csv.getFile('products.csv');\n\nconst translator = new GoogleTranslateHelper(process.env.GOOGLE_CLOUD_API_KEY!, 'en', 'ru');\n\nlet translateProducts$ = src.selectRx().pipe(\n    translator.operator(),\n    log()\n);\nawait run(translateProducts$);\n```\n\n ### HttpClientHelper\n\nThis class help you to make requests to http and https resources and get data from it.\n\nMethods:\n\n```typescript\n    // Create helper object\n    // baseUrl: this string will be used as start part of urls in any helper methods\n    // headers: will be added to headers in all requests maden with this helper instance\n    constructor(baseUrl?: string, headers?: Record\u003cstring, string\u003e);\n\n    // GET request\n\n    async get(url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cResponse\u003e;\n    async getJson(url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cany\u003e;\n    async getText(url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cstring\u003e;\n    async getBlob(url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cBlob\u003e;\n    async getFileContents(url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cBlob\u003e;\n\n    getJsonOperator\u003cT, R = T\u003e(): OperatorFunction\u003cT, R\u003e;\n    getJsonOperator\u003cT, R = T\u003e(url: string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n    getJsonOperator\u003cT, R = T\u003e(getUrl: (value: T) =\u003e string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n\n    getTextOperator\u003cT\u003e(): OperatorFunction\u003cT, string\u003e;\n    getTextOperator\u003cT\u003e(url: string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, string\u003e;\n    getTextOperator\u003cT\u003e(getUrl: (value: T) =\u003e string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, string\u003e;\n\n    getBlobOperator\u003cT\u003e(): OperatorFunction\u003cT, Blob\u003e;\n    getBlobOperator\u003cT, R = T\u003e(url: string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n    getBlobOperator\u003cT, R = T\u003e(getUrl: (value: T) =\u003e string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n\n    getFileContentsOperator\u003cT\u003e(): OperatorFunction\u003cT, Blob\u003e;\n    getFileContentsOperator\u003cT, R = T\u003e(url: string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n    getFileContentsOperator\u003cT, R = T\u003e(getUrl: (value: T) =\u003e string, toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n\n    // POST request\n\n    async post(body: string, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cResponse\u003e;\n    async postJson(body: any, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cany\u003e;\n    async postText(body: string, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cstring\u003e;\n\n    postJsonOperator\u003cT, R = T\u003e(bodyParam?: any | ((value: T) =\u003e any), urlParam?: string | ((value: T) =\u003e string), toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n\n    // PUT request\n\n    async put(body: string, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cResponse\u003e;\n    async putJson(body: any, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cany\u003e;\n    async putText(body: string, url?: string, headers?: Record\u003cstring, string\u003e): Promise\u003cstring\u003e;\n\n    putJsonOperator\u003cT, R = T\u003e(bodyParam?: any | ((value: T) =\u003e any), urlParam?: string | ((value: T) =\u003e string), toProperty?: string, headers?: Record\u003cstring, string\u003e): OperatorFunction\u003cT, R\u003e;\n\n    // Simple fetch method\n\n    async fetch(url: string): Promise\u003cResponse\u003e;\n    async fetch(url: string, method: 'GET' | 'POST' | 'PUT' | 'DELETE', headers: Record\u003cstring, string\u003e): Promise\u003cResponse\u003e;\n    async fetch(url: string, init: RequestInit): Promise\u003cResponse\u003e;\n```\n\nExample:\n\n```typescript\nimport { Csv, HttpClientHelper, run } from \"etl-gun\";\nimport { map } from \"rxjs\";\n\nlet csv = new Csv.Endpoint();\nlet src = csv.getFile('products.csv');\n\nconst mySite = new HttpClientHelper('http://www.mysite.com');\n\nlet sendProductsToSite$ = src.selectRx().pipe(\n    map(p =\u003e mySite.post(p)),\n);\nawait run(sendProductsToSite$);\n ```\n\n### Header\n\nThis class can store array of column names and convert object to array or array to object representation.\n\n```typescript\nimport { Postgres, Csv, Header, log, push, run } from \"etl-gun\";\nimport { map } from \"rxjs\";\n\nconst pg = new Postgres.Endpoint(\"postgres://user:password@127.0.0.1:5432/database\");\nconst source = pg.getTable(\"users\");\n\nlet csv = new Csv.Endpoint();\nconst dest = csv.getFile(\"users.csv\");\nconst header = new Header([{\"id\": \"number\"}, \"name\", {\"login\": \"string\", nullValue: \"-null-\"}, \"email\"]);\n\nlet sourceToDest$ = source.selectRx().pipe(\n    map(v =\u003e header.objToArr(v)),\n    push(dest)\n);\nawait run(sourceToDest$);\n ```\n\n### Utility functions\n\nThis functions implements some useful things to manipulate data.\n\n```typescript\n// Stop thread for the specified in milliseconds delay.\nasync function wait(delay: number): Promise\u003cvoid\u003e;\n// Join url parts (or path parts) to full url (or path) with delimeter\nfunction pathJoin(parts: string[], sep: string = '/'): string;\n// Extract 'html' from '/var/www/html'\nfunction extractFileName(path: string): string;\n// Extract '/var/www' from '/var/www/html'\nfunction extractParentFolderPath(path: string): string\n// Get object part by json path\nfunction getByJsonPath(obj: {}, jsonPath?: string): any;\n// Get child element of array or object by element property value\nfunction getChildByPropVal(obj: {}, propName: string, propVal?: any): any;\n// Convert object to string\nfunction dumpObject(obj: any, deep: number = 1): string;\n// Get child by it's property value\n// For example getChildByPropVal([{prop1: 'val1'}, {prop1: 'val2'}], 'prop1', 'val1') -\u003e returns {prop1: 'val1'}\nfunction getChildByPropVal(obj: {} | [], propName: string, propVal?: any): any;\n ```\n\n------------------------------------------------------------------------------------------------------------------------------------------------------------------------\n\n# License\n\nThis library is provided with [MIT](LICENSE) license.\n\nThe detailed list of Open Source dependencies can be found in [Fossa report](https://app.fossa.com/reports/e0347ddb-3a0e-4c88-8655-0ec072928611).\n\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Ftop-guns%2Fetl-gun?ref=badge_large)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftop-guns%2Fetl-gun","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftop-guns%2Fetl-gun","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftop-guns%2Fetl-gun/lists"}