{"id":13617183,"url":"https://github.com/wgzhao/Addax","last_synced_at":"2025-04-14T03:33:08.857Z","repository":{"id":37015335,"uuid":"197400272","full_name":"wgzhao/Addax","owner":"wgzhao","description":"A fast and versatile ETL tool that can transfer data between RDBMS and NoSQL seamlessly","archived":false,"fork":false,"pushed_at":"2025-04-11T07:28:41.000Z","size":46063,"stargazers_count":1257,"open_issues_count":5,"forks_count":309,"subscribers_count":34,"default_branch":"master","last_synced_at":"2025-04-12T00:02:54.270Z","etag":null,"topics":["clickhouse","database","etl","excel","hadoop","hdfs","hive","impala","influxdb","kudu","mysql","oracle","postgresql","sqlserver","trino"],"latest_commit_sha":null,"homepage":"https://wgzhao.github.io/Addax/","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wgzhao.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":"SECURITY.md","support":"support_data_sources.md","governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-07-17T13:58:01.000Z","updated_at":"2025-04-11T07:28:44.000Z","dependencies_parsed_at":"2023-02-10T19:01:11.630Z","dependency_job_id":"2fc31999-9d4b-46e8-beb9-7f7947e23e16","html_url":"https://github.com/wgzhao/Addax","commit_stats":{"total_commits":1306,"total_committers":25,"mean_commits":52.24,"dds":0.1891271056661562,"last_synced_commit":"821bc0e0fe6f780fe91987f4e000749ff4e82d42"},"previous_names":["wgzhao/datax"],"tags_count":53,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgzhao%2FAddax","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgzhao%2FAddax/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgzhao%2FAddax/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wgzhao%2FAddax/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wgzhao","download_url":"https://codeload.github.com/wgzhao/Addax/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248815784,"owners_count":21165981,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","database","etl","excel","hadoop","hdfs","hive","impala","influxdb","kudu","mysql","oracle","postgresql","sqlserver","trino"],"created_at":"2024-08-01T20:01:38.010Z","updated_at":"2025-04-14T03:33:08.851Z","avatar_url":"https://github.com/wgzhao.png","language":"Java","funding_links":[],"categories":["Java","数据管道","大数据","Integrations"],"sub_categories":["ETL and Data Processing"],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"Addax Logo\" src=\"https://github.com/wgzhao/Addax/blob/master/docs/images/logo.svg?raw=true\" width=\"205\" /\u003e\n\u003c/p\u003e\n\u003ch1 align=\"center\"\u003eAddax\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    \u003cb\u003eA versatile open-source ETL tool\u003c/b\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\nAddax is an extensible ETL (Extract, Transform, Load) tool supporting over 20 SQL/NoSQL data sources, developed as a fork and evolution of Alibaba's \u003ca href=\"https://github.com/alibaba/datax\"\u003eDataX\u003c/a\u003e. It provides a growing ecosystem of plugins and offers easy-to-follow configuration for data integrations.\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n   \u003ca href=\"https://github.com/wgzhao/Addax/releases\"\u003e\n      \u003cimg src=\"https://img.shields.io/github/release/wgzhao/addax.svg\" alt=\"Release Version\" /\u003e\n   \u003c/a\u003e\n   \u003ca href=\"https://github.com/wgzhao/Addax/workflows/Maven%20Package/badge.svg\"\u003e\n       \u003cimg src=\"https://github.com/wgzhao/Addax/workflows/Maven%20Package/badge.svg\" alt=\"Maven Package\" /\u003e\n   \u003c/a\u003e\n\u003c/p\u003e\n\n[简体中文](README_zh.md)\n\n---\n\n## 🚀 Features\n\n- Supports 20+ SQL and NoSQL data sources, and easily extendable for more.\n- Configurable via simple JSON-based job descriptions.\n- Actively maintained with improved architecture and added functionality compared to [DataX](https://github.com/alibaba/datax).\n- Docker images for quick deployment.\n\n---\n\n## 📚 Documentation\n\nDetailed instructions on installation, configuration, and usage are available:\n\n- [Online Documentation](https://wgzhao.github.io/Addax/)\n- [GitHub Docs](https://wgzhao.github.io/Addax)\n- [Plugin Comparison](difference.md)\n\n---\n\n## 📦 Supported Data Sources\n\nAddax supports a wide range of database systems and file sources. Below is a selection of supported platforms:\n\n\u003ctable\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/cassandra.svg\" height=\"50px\" alt=\"Cassandra\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/clickhouse.svg\" height=\"50px\" alt=\"Clickhouse\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/databend.svg\" height=\"50px\" alt=\"DataBend\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/db2.svg\" height=\"50px\" alt=\"IMB DB2\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/dbase.svg\" height=\"50px\" alt=\"dBase\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/doris.svg\"  height=\"50px\" alt=\"Doris\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/elasticsearch.svg\" height=\"50px\" alt=\"Elasticsearch\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/excel.svg\" height=\"50px\" alt=\"Excel\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/greenplum.svg\" height=\"50px\" alt=\"Greenplum\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/hbase.svg\" height=\"50px\" alt=\"Apache HBase\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/hive.svg\" height=\"50px\" alt=\"Hive\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/influxdata.svg\" height=\"50px\" alt=\"InfluxDB\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/kafka.svg\" height=\"50px\" alt=\"Kafka\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/kudu.svg\" height=\"50px\" alt=\"Kudu\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/minio.svg\" height=\"50px\" alt=\"MinIO\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/mongodb.svg\" height=\"50px\" alt=\"MongoDB\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/mysql.svg\" height=\"50px\" alt=\"MySQL\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/oracle.svg\" height=\"50px\" alt=\"Oracle\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/phoenix.svg\" height=\"50px\" alt=\"Phoenix\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/postgresql.svg\" height=\"50px\" alt=\"PostgreSQL\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/presto.svg\" height=\"50px\" alt=\"Presto\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/redis.svg\" height=\"50px\" alt=\"Redis\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/s3.svg\" height=\"50px\" alt=\"Amazon S3\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/sqlite.svg\" height=\"50px\" alt=\"SQLite\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/sqlserver.svg\" height=\"50px\" alt=\"SQLServer\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/starrocks.svg\" height=\"50px\" alt=\"Starrocks\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/sybase.svg\" height=\"50px\" alt=\"Sybase\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/tdengine.svg\" height=\"50px\" alt=\"TDengine\"  style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/trino.svg\" height=\"50px\" alt=\"Trino\" style=\"border: 1px solid #ddd;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/access.svg\" height=\"50px\" alt=\"Access\" style=\"border: 1px solid #add;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/sap.svg\" height=\"50px\" alt=\"SAP HANA\" style=\"border: 1px solid #add;\"\u003e\u003c/td\u003e\n\u003ctd\u003e\u003cimg src=\"./docs/images/logos/paimon.svg\" height=\"50px\" alt=\"Paimon\" style=\"border: 1px solid #add;\"\u003e\u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e **See the [full list](support_data_sources.md) of supported data sources**.\n\n---\n\n## 🛠️ Getting Started\n\nAddax can be quickly installed and used via Docker, installation scripts, or compiled from source.\n\n### 1. Use docker image\n\nPull the prebuilt Docker image and run a test job:\n\n```shell\ndocker pull quay.io/wgzhao/addax:latest\ndocker run -ti --rm --name addax \\\n  quay.io/wgzhao/addax:latest \\\n  /opt/addax/bin/addax.sh /opt/addax/job/job.json\n```\n\n### 2. Use installation script\n\nInstall Addax with a single command:\n\n```bash\n/bin/bash -c \"$(curl -fsSL https://raw.githubusercontent.com/wgzhao/Addax/master/install.sh)\"\n```\n\n\u003e Installation paths: /usr/local (macOS), /opt/addax/ (Linux).\n\n### 3. Compile and Package from Source\n\nFor developers targeting custom-builds, compile and package Addax locally:\n\n```shell\ngit clone https://github.com/wgzhao/addax.git addax\ncd addax\nexport MAVEN_OPTS=\"-DskipTests -Dmaven.javadoc.skip=true -Dmaven.source.skip=true -Dgpg.skip=true\"\nmvn clean package\nmvn package -Pdistribution\n```\n\nThe compiled binary will be in `target/addax-\u003cversion\u003e`.\n\n### 4. Run Your First Task\n\nLoad sample job configuration and test the setup:\n\n```bash\nbin/addax.sh job/job.json\n```\n\nExplore more [example jobs](docs/assets/jobs)\n\n---\n\n## 📖 Runtime Requirements\n\n- **Java Runtime**: JDK 1.8+\n- **Python Version**: Python 2.7+ / 3.7+ (Windows only)\n\n---\n\n## 🧩 Developing Addax\n\n### Building Documentation\n\nGenerate or preview online documentation locally using [mkdocs](https://www.mkdocs.org):\n\n```shell\npython3 -m pip install mkdocs-material\nmkdocs build\nmkdocs serve -a 0.0.0.0:8888\n```\n\nFor publishing:\n\n```shell\nexport version=4.1.5\ngit checkout $version\nmike deploy -p $version\n```\n\n---\n\n## Code Style Guidelines\n\nFollow general Java conventions and patterns:\n\n1. Use IntelliJ IDE with [Airlift's Code Style](https://github.com/airlift/codestyle)\n2. Categorize exceptions clearly with AddaxException (e.g., `AddaxException(REQUIRE_VALUE, \"missing required parameter\")`).\n3. Use the Java 8 Stream API cautiously (avoid in performance-sensitive areas).\n4. Avoid ternary operators for non-trivial expressions.\n5. Include proper Apache license headers in every file.\n\n\u003e Refer to our [Programming Guidelines](https://cbea.ms/git-commit/) for commit message formats.\n\n---\n\n## 🗓️ Versioning Scheme\n\nThis project adheres to the [Semantic Versioning (SemVer)](https://semver.org/) standard with the format `x.y.z`. The meanings of each segment are as follows:\n\n- **z（Patch Version）**:\n    - Bug fixes and performance improvements that do not affect compatibility with existing features.\n    - Example: `1.2.3 → 1.2.4`\n\n- **y（Minor Version）**:\n    - Introducing new features or module adjustments that could break backward compatibility.\n    - Example: `1.2.3 → 1.3.0`\n\n- **x（Major Version）**:\n    - Significant changes or new features that are often incompatible with previous versions.\n    - Example: `1.3.0 → 2.0.0`\n\n---\n\n## 🌟 Star History\n\n## Star History\n\n\u003ca href=\"https://www.star-history.com/#wgzhao/Addax\u0026Date\"\u003e\n \u003cpicture\u003e\n   \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://api.star-history.com/svg?repos=wgzhao/Addax\u0026type=Date\u0026theme=dark\" /\u003e\n   \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"https://api.star-history.com/svg?repos=wgzhao/Addax\u0026type=Date\" /\u003e\n   \u003cimg alt=\"Star History Chart\" src=\"https://api.star-history.com/svg?repos=wgzhao/Addax\u0026type=Date\" /\u003e\n \u003c/picture\u003e\n\u003c/a\u003e\n\n--- \n\n## ⚖️ License\n\nThis software is free to use under the [Apache License 2.0](/LICENSE).\n\n---\n\n## 💌 Special Thanks\n\nSpecial thanks to [JetBrains](https://jb.gg/OpenSource) for providing open-source support to this project.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgzhao%2FAddax","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwgzhao%2FAddax","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwgzhao%2FAddax/lists"}