{"id":22235026,"url":"https://github.com/pzaino/microetl","last_synced_at":"2025-07-14T18:13:56.864Z","repository":{"id":182338556,"uuid":"668298302","full_name":"pzaino/microETL","owner":"pzaino","description":"A simple, reusable, templates based ETL (Extract, Transform and Load) library and framework written in Python","archived":false,"fork":false,"pushed_at":"2025-04-04T06:02:59.000Z","size":392,"stargazers_count":1,"open_issues_count":8,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-04T08:11:18.334Z","etag":null,"topics":["data-migration","data-migration-tool","data-migrator","data-transformation","etl","etl-pipeline","etl-process","framework","python3"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pzaino.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-19T13:30:28.000Z","updated_at":"2025-03-21T19:48:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"5f1c6409-b4bd-41cc-8004-b32f8ad35657","html_url":"https://github.com/pzaino/microETL","commit_stats":null,"previous_names":["pzaino/microetl"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pzaino/microETL","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzaino%2FmicroETL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzaino%2FmicroETL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzaino%2FmicroETL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzaino%2FmicroETL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pzaino","download_url":"https://codeload.github.com/pzaino/microETL/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pzaino%2FmicroETL/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265328100,"owners_count":23747840,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-migration","data-migration-tool","data-migrator","data-transformation","etl","etl-pipeline","etl-process","framework","python3"],"created_at":"2024-12-03T02:11:22.856Z","updated_at":"2025-07-14T18:13:56.846Z","avatar_url":"https://github.com/pzaino.png","language":"Python","readme":"# microETL\n\nA library and framework for data transformation, ETL (Extract, Transform and Load) and data migration using multiple data sources, DSL and multiple data destinations.\n\n## Installation\n\nClone this repo and then run\n\n```bash\ncd microETL\npip install -r requirements.txt\n```\n\nAfter that, make sure your copy of jinjaSQL is using markupsafe library. To do that:\n\n(on the mac):\n\n```bash\nvi $(homebrew_prefix)/lib/python3.9/site-packages/jinjasql/core.py\n```\n\n(on MS Windows, using VSCode)\n\n```powershell\ncode C:\\Users\\\u003cyour-user-name\u003e\\AppData\\Local\\Programs\\Python\\Python\u003cver\u003e\\Lib\\site-packages\\jinjasql\\core.py\n```\n\nWhere\n\n* `\u003cyour-user-name\u003e` is (obviously) your Windows username\n* `\u003cver\u003e` is your Python version (for example 311 for python 3.11)\n\nAnd, around line 6, if you see this import:\n\n```python\nfrom jinja2.utils import Markup\n```\n\ncomment it out and immediately after, add this:\n\n```python\nfrom markupsafe import Markup\n```\n\n## Usage\n\nTo learn everything about using microETL, please check the [docs](docs/README.md).\n\n## Contributing\n\nMicroETL is released under CDDL v1.1 license, so everyone can contribute to make it better. Before contributing, please read the [docs](docs/README.md) to make sure you fully understand all the goals and so your contributions are always in-line with the project goals, thank you.\n\n## License\n\nCDDL v1.1\n\nIf you're not familiar with the CDDL v1.1 license, please check [this](https://fossa.com/blog/open-source-licenses-101-cddl-common-development-distribution-license/) link for more \"human readable\" info.\n\nIn very short terms the CDDL v1.1 is a weak copyleft license which allows you to link my work with copyrighted source, however, if you modify my work (or improve it) you still need to release your changes under CDDL v1.1 and therefore make your changes available to everyone to benefit. That does NOT include your software that may be using my work, only the improvements you've made to my work.\n\nAlso, obviously, you cannot claim it's your work, you must always give credit to the original author (me) and all the folks that have contributed to this project.\n\n## Acknowledgements\n\nGiven that MicroETL is mostly a \"process\" that uses other libraries, I would like to thank the authors of the following libraries:\n\n* JinjaSQL\n* PyYAML\n* Jinja2\n* MarkupSafe\n* SQLAlchemy\n* JSON\n* Pandas\n* Numpy\n* JSONBender\n* Snowflake client\n* MySQL client\n* PostgreSQL client\n* SQLite client\n* Neo4J client\n* MongoDB client\n* JSONSchema\n\nI hope I have included them all, if I missed any, please let me know and I will add them to the list, thank you!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpzaino%2Fmicroetl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpzaino%2Fmicroetl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpzaino%2Fmicroetl/lists"}