{"id":13437563,"url":"https://github.com/getdozer/dozer","last_synced_at":"2025-04-11T06:26:15.295Z","repository":{"id":65307375,"uuid":"531051081","full_name":"getdozer/dozer","owner":"getdozer","description":"Dozer is a real-time data movement tool that leverages CDC from various sources and moves data into various sinks. ","archived":false,"fork":false,"pushed_at":"2024-06-18T22:23:39.000Z","size":62176,"stargazers_count":1541,"open_issues_count":144,"forks_count":125,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-03T17:08:41.707Z","etag":null,"topics":["api","apis","clickhouse","data","datawarehouse","debe","etl","low-code","postgres","realtime","rust","snowflake","sql","streaming"],"latest_commit_sha":null,"homepage":"https://getdozer.io","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/getdozer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-31T11:20:38.000Z","updated_at":"2025-03-28T13:07:34.000Z","dependencies_parsed_at":"2023-10-16T18:54:25.940Z","dependency_job_id":"ee9d5ac5-980f-4518-a07d-b52748d9e5c5","html_url":"https://github.com/getdozer/dozer","commit_stats":null,"previous_names":[],"tags_count":46,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getdozer%2Fdozer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getdozer%2Fdozer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getdozer%2Fdozer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/getdozer%2Fdozer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/getdozer","download_url":"https://codeload.github.com/getdozer/dozer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248353392,"owners_count":21089623,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api","apis","clickhouse","data","datawarehouse","debe","etl","low-code","postgres","realtime","rust","snowflake","sql","streaming"],"created_at":"2024-07-31T03:00:58.344Z","updated_at":"2025-04-11T06:26:15.261Z","avatar_url":"https://github.com/getdozer.png","language":"Rust","readme":"## Overview\n\nDozer is a **real time data movement tool leveraging CDC from various sources to multiple sinks.**\n\nDozer is magnitudes of times faster than Debezium+Kafka and natively supports stateless transformations. \nPrimarily used for moving data into warehouses. In our own application, we move data to **Clickhouse** and build data APIs and integration with LLMs. \n\n## How to use it\nDozer runs with a single configuration file like the following:\n```yaml\napp_name: dozer-bench\nversion: 1\nconnections:\n  - name: pg_1\n    config: !Postgres\n      user: user\n      password: postgres\n      host: localhost\n      port: 5432\n      database: customers\nsinks:\n  - name: customers\n    config: !Dummy\n      table_name: customers\n```\n\nFull documentation can be found [here](https://github.com/getdozer/dozer/blob/main/dozer-types/src/models/config.rs#L15)\n\n\n## Supported Sources\n\n| Connector            | Extraction | Resuming | Enterprise          |\n| -------------------- | ---------- | -------- | ------------------- |\n| Postgres             | ✅          | ✅        | ✅                   |\n| MySQL                | ✅          | ✅        | ✅                   |\n| Snowflake            | ✅          | ✅        | ✅                   |\n| Kafka                | ✅          | 🚧        | ✅                   |\n| MongoDB              | ✅          | 🎯        | ✅                   |\n| Amazon S3            | ✅          | 🎯        | ✅                   |\n| Google Cloud Storage | ✅          | 🎯        | ✅                   |\n| **Oracle             | ✅          | ✅        | **Enterprise Only** |\n| **Aerospike          | ✅          | ✅        | **Enterprise Only** |\n\n\n## Supported Sinks\n| Database   | Connectivity | Enterprise          |\n| ---------- | ------------ | ------------------- |\n| Clickhouse | ✅            |                     |\n| Postgres   | ✅            |                     |\n| MySQL      | ✅            |                     |\n| Big Query  | ✅            |                     |\n| Oracle     | ✅            | **Enterprise Only** |\n| Aerospike  | ✅            | **Enterprise Only** |","funding_links":[],"categories":["HarmonyOS","Rust","api","Integrations"],"sub_categories":["Windows Manager","Data Transfer and Synchronization"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetdozer%2Fdozer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgetdozer%2Fdozer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgetdozer%2Fdozer/lists"}