https://github.com/flow-php/etl-adapter-amphp
PHP ETL AMP asynchronous processing adapter
https://github.com/flow-php/etl-adapter-amphp
amphp data-engineering data-processing etl flow-php
Last synced: 23 days ago
JSON representation
PHP ETL AMP asynchronous processing adapter
- Host: GitHub
- URL: https://github.com/flow-php/etl-adapter-amphp
- Owner: flow-php
- License: mit
- Archived: true
- Created: 2022-04-26T20:09:39.000Z (almost 4 years ago)
- Default Branch: 1.x
- Last Pushed: 2023-11-05T13:04:33.000Z (over 2 years ago)
- Last Synced: 2026-01-16T00:20:17.625Z (2 months ago)
- Topics: amphp, data-engineering, data-processing, etl, flow-php
- Language: PHP
- Homepage:
- Size: 136 KB
- Stars: 4
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-amphp - flow-php/etl-adapter-amphp - AMP asynchronous processing provider for [Flow PHP](https://github.com/flow-php/flow) (Unsorted yet / Tunnel)
README
# Flow Adapter: [AMP](https://amphp.org/)
Flow PHP's Adapter AmpHP is a meticulously crafted library aimed at integrating the asynchronous capabilities of AmpHP
within your ETL (Extract, Transform, Load) workflows. This adapter is instrumental for developers aspiring to execute
non-blocking data operations, thereby optimizing performance and responsiveness in their data transformation workflows.
By leveraging the Adapter AmpHP library, developers can access a robust suite of features engineered for precise
asynchronous data operations, simplifying complex data transformations while enhancing operational efficiency. The
Adapter AmpHP library encapsulates a rich set of functionalities, offering a streamlined API for managing asynchronous
tasks, which is indispensable in modern data processing and transformation landscapes. This library mirrors Flow PHP's
commitment to delivering versatile and efficient data processing solutions, making it a superior choice for developers
dealing with asynchronous operations in large-scale and data-intensive environments. With Flow PHP's Adapter AmpHP,
embracing asynchronous data processing within your ETL workflows becomes a seamless and efficient endeavor, aligning
harmoniously with the robust and adaptable framework of the Flow PHP ecosystem.
Following communication protocols are supported:
- TCP/IP (only local) - `127.0.0.1:6651`
- Unix Domain Socket - `uinx:///var/run/etl.sock`
# Installation
```
composer require flow-php/etl-adapter-amphp:1.x@dev
```
Example usage:
```php
pushHandler(new StreamHandler("php://stdout", LogLevel::DEBUG, false));
$logger->pushHandler(new StreamHandler("php://stderr", LogLevel::ERROR, false));
(new Flow)
->read(new CSVExtractor(
$path = __DIR__ . '/data/dataset.csv',
10_000,
0
))
->pipeline(
new LocalSocketPipeline(
SocketServer::unixDomain(__DIR__ . "/var/run/", $logger),
new ChildProcessLauncher(__DIR__ . "/vendor/bin/worker-amp", $logger),
$workers = 8
)
)
->withEntry('id', ref('id')->cast('int'))
->withEntry('name', concat(ref('name'), lit(' '), ref('last name')))
->drop('last_name')
->load(new DbalLoader($tableName, $dbConnectionParams))
->run();
```
This adapter comes with built-in [worker](bin/worker-amp) CLI application
but feel free to create custom.
Customization of the works will let you adjust logger or autoloader.