https://github.com/ddeutils/ddeutil-extensions
:building_construction: Dynamic data processing & transformation plugins
https://github.com/ddeutils/ddeutil-extensions
data-processing data-transformation
Last synced: 4 months ago
JSON representation
:building_construction: Dynamic data processing & transformation plugins
- Host: GitHub
- URL: https://github.com/ddeutils/ddeutil-extensions
- Owner: ddeutils
- License: mit
- Created: 2024-08-06T15:13:00.000Z (10 months ago)
- Default Branch: main
- Last Pushed: 2025-02-18T16:04:09.000Z (4 months ago)
- Last Synced: 2025-02-18T16:23:38.445Z (4 months ago)
- Topics: data-processing, data-transformation
- Language: Python
- Homepage:
- Size: 78.1 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Extensions
[](https://github.com/ddeutils/ddeutil-extensions)
[](https://github.com/ddeutils/ddeutil-extensions/blob/main/LICENSE)
[](https://github.com/psf/black)An **extensions functions and objects** which provides all plug-ins and objects
that use for data processing and transformation.## :round_pushpin: Installation
```shell
pip install -U ddeutil-extensions
```## :rocket: Features
This extensions package provides 3 main components:
- Plug-in the extension tasks that use with the [Workflow](https://github.com/ddeutils/ddeutil-extensions)
- Connection and Dataset interface objects
- Schema models### Connection
The connection for worker able to do anything.
```yaml
conn_postgres_data:
type: conn.Postgres
url: 'postgres//username:${ENV_PASS}@hostname:port/database?echo=True&time_out=10'
``````python
from ddeutil.extensions.conn import Connconn = Conn.from_loader(name='conn_postgres_data', externals={})
assert conn.ping()
```### Dataset
The dataset is defined any objects on the connection. This feature was implemented
on `/extensions` because it has a lot of tools that can interact with any data systems
in the data tool stacks.```yaml
ds_postgres_customer_tbl:
type: dataset.PostgresTbl
conn: 'conn_postgres_data'
features:
id: serial primary key
name: varchar( 100 ) not null
``````python
from ddeutil.extensions.datasets.pg import PostgresTbldataset = PostgresTbl.from_loader(name='ds_postgres_customer_tbl', externals={})
assert dataset.exists()
```## :speech_balloon: Contribute
I do not think this project will go around the world because it has specific propose,
and you can create by your coding without this project dependency for long term
solution. So, on this time, you can open [the GitHub issue on this project :raised_hands:](https://github.com/ddeutils/ddeutil-extensions/issues)
for fix bug or request new feature if you want it.