https://github.com/barlou/tools
Reusable Python tools for data engineering pipelines — cloud storage client (AWS S3, OVH), structured logging with cloud flush strategies, and Hive-partitioned Parquet/ORC archiving. Built for Airflow tasks and RL training workloads.
https://github.com/barlou/tools
airflow aws-s3 cloud-storage data-engineering github-actions logging orc ovh parquet python
Last synced: about 1 month ago
JSON representation
Reusable Python tools for data engineering pipelines — cloud storage client (AWS S3, OVH), structured logging with cloud flush strategies, and Hive-partitioned Parquet/ORC archiving. Built for Airflow tasks and RL training workloads.
- Host: GitHub
- URL: https://github.com/barlou/tools
- Owner: barlou
- License: other
- Created: 2026-03-27T14:22:30.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2026-05-03T13:56:50.000Z (about 1 month ago)
- Last Synced: 2026-05-03T14:09:08.024Z (about 1 month ago)
- Topics: airflow, aws-s3, cloud-storage, data-engineering, github-actions, logging, orc, ovh, parquet, python
- Language: Python
- Homepage:
- Size: 48.8 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files: