https://github.com/fdmorison/tiozin
Tiozin, your friendly ETL framework
https://github.com/fdmorison/tiozin
data declarative etl framework pipeline
Last synced: about 2 months ago
JSON representation
Tiozin, your friendly ETL framework
- Host: GitHub
- URL: https://github.com/fdmorison/tiozin
- Owner: fdmorison
- License: mpl-2.0
- Created: 2025-11-30T16:58:10.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2026-04-20T01:42:04.000Z (2 months ago)
- Last Synced: 2026-04-20T02:42:12.992Z (2 months ago)
- Topics: data, declarative, etl, framework, pipeline
- Language: Python
- Homepage: https://pypi.org/project/tiozin/
- Size: 10.3 MB
- Stars: 6
- Watchers: 0
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: docs/CONTRIBUTING.md
- License: LICENSE
- Agents: agents.md
Awesome Lists containing this project
README
# Tiozin
---
ETL shouldn't require 80 files, 50 YAMLs, and a PhD in complexity.
Tiozin brings it back to basics: **Transform. Input. Output.** Nothing more, nothing less.
A lightweight Python framework that makes data jobs declarative, testable, and actually enjoyable to write.
## Philosophy
Your uncle's advice: keep it simple, readable, and testable.
Tiozin is built around a small set of principles that are not features but constraints that shape the design.
- **Declarative**: Define what, not how
- **Pluggable**: Swap runners, registries, and plugins when needed
- **Metadata-native**: Execution and metadata walk together
- **Observable**: Logs that actually help
- **Testable**: Mock anything, validate everything
## One-Minute Example
```bash
pip install tiozin
```
**Define a declarative job**
```yaml
kind: LinearJob
name: example_job
owner: tiozin@tiozin.com
maintainer: tiozin
cost_center: tio_scrooge
org: tiozin
region: latam
domain: marketing
subdomain: campaigns
layer: refined
product: users
model: customers
runner:
kind: NoOpRunner
name: postgres
log_level: "{{ ENV.LOG_LEVEL }}"
db_url: postgres://user:{{ SECRET.FAKE_PASSWORD }}@host:5432/dbname
inputs:
- kind: NoOpInput
name: load_it
layer: raw
path: .output/lake-{{domain}}-{{layer}}/{{product}}/{{model}}/date={{ DAY[-1] }}
transforms:
- kind: NoOpTransform
name: process_it
strategy: sha256
outputs:
- kind: NoOpOutput
name: save_it
path: .output/lake-{{domain}}-{{layer}}/{{product}}/{{model}}/{{ today }}
```
Run it:
```bash
$ tiozin run examples/jobs/dummy.yaml
```
**Using Python directly**
```python
from tiozin import TiozinApp
app = TiozinApp()
app.run("examples/jobs/dummy.yaml")
```
Done. No ceremony, no boilerplate.
## Who is Tiozin for?
Tiozin is human-readable and machine-generatable:
- Data engineers who want reusable pipeline components
- Teams that value declarative jobs
- Projects that require testable ETL logic
- Pipelines where metadata is connected to the execution model
- Teams leveraging AI agents 🤖 to author and maintain data jobs
## Documentation
### Writing Declarative Jobs
- [Quick Start](https://github.com/fdmorison/tiozin/blob/main/docs/quick-start.md)
- [Working with Jobs](https://github.com/fdmorison/tiozin/blob/main/docs/working-with-jobs.md)
- [Examples](https://github.com/fdmorison/tiozin/blob/main/docs/examples.md)
### Concepts
- [Family Model: Tios and Tiozins](https://github.com/fdmorison/tiozin/blob/main/docs/concepts/family.md)
- [Jobs](https://github.com/fdmorison/tiozin/blob/main/docs/concepts/jobs.md)
- [Runners](https://github.com/fdmorison/tiozin/blob/main/docs/concepts/runners.md)
- [Inputs, Transforms & Outputs](https://github.com/fdmorison/tiozin/blob/main/docs/concepts/steps.md)
- [Registries](https://github.com/fdmorison/tiozin/blob/main/docs/concepts/registries.md)
### Extending Tiozin
- [Creating a Provider Family](https://github.com/fdmorison/tiozin/blob/main/docs/extending/families.md)
- [Creating Pluggable Tiozins](https://github.com/fdmorison/tiozin/blob/main/docs/extending/tiozins.md)
- [Tio Proxy: Adding Cross Cutting Family Features](https://github.com/fdmorison/tiozin/blob/main/docs/extending/proxies.md)
- [Tio Kernel - The Core Family](https://github.com/fdmorison/tiozin/blob/main/docs/tio_kernel/index.md)
- [Tio Spark - Spark Family Example](https://github.com/fdmorison/tiozin/blob/main/docs/tio_spark/index.md)
- [Tio Duckdb - DuckDB Family Example](https://github.com/fdmorison/tiozin/blob/main/docs/tio_duckdb/index.md)
### How-to Guides
- [How to Configure OpenLineage](https://github.com/fdmorison/tiozin/blob/main/docs/how-to/openlineage.md)
- [How to Use Schemas in Jobs](https://github.com/fdmorison/tiozin/blob/main/docs/how-to/schemas.md)
- [How to Use Secrets in Jobs](https://github.com/fdmorison/tiozin/blob/main/docs/how-to/secrets.md)
### Reference
- [Settings Reference: tiozin.yaml](https://github.com/fdmorison/tiozin/blob/main/docs/settings.md)
- [Templates Reference](https://github.com/fdmorison/tiozin/blob/main/docs/templates.md)
- [API Reference](https://github.com/fdmorison/tiozin/blob/main/docs/api.md)
## Contributing
Contributions are welcome! See [CONTRIBUTING.md](https://github.com/fdmorison/tiozin/blob/main/docs/CONTRIBUTING.md) for guidelines.
## License
This project is licensed under the [Mozilla Public License 2.0](LICENSE).