https://github.com/mobiletelesystems/syncmaster

Low-code ETL tool, based on onETL. WIP
https://github.com/mobiletelesystems/syncmaster

etl low-code transfer-data

Last synced: about 2 months ago
JSON representation

Low-code ETL tool, based on onETL. WIP

• Host: GitHub
• URL: https://github.com/mobiletelesystems/syncmaster
• Owner: MobileTeleSystems
• License: apache-2.0
• Created: 2024-02-20T09:29:16.000Z (11 months ago)
• Default Branch: develop
• Last Pushed: 2024-10-29T09:35:44.000Z (2 months ago)
• Last Synced: 2024-10-29T11:44:29.002Z (2 months ago)
• Topics: etl, low-code, transfer-data
• Language: Python
• Homepage: https://syncmaster.readthedocs.io
• Size: 1.68 MB
• Stars: 5
• Watchers: 4
• Forks: 0
• Open Issues: 3
• Metadata Files:
  • Readme: README.rst
  • Contributing: CONTRIBUTING.rst
  • License: LICENSE.txt
  • Security: SECURITY.rst

Awesome Lists containing this project

README

        
.. _readme:

Data.SyncMaster

===============

|Repo Status| |PyPI| |PyPI License| |PyPI Python Version| |Docker image| |Documentation|

|Build Status| |Coverage|  |pre-commit.ci|

.. |Repo Status| image:: https://www.repostatus.org/badges/latest/wip.svg

    :target: https://www.repostatus.org/#wip

.. |PyPI| image:: https://img.shields.io/pypi/v/data-syncmaster

    :target: https://pypi.org/project/data-syncmaster/

.. |PyPI License| image:: https://img.shields.io/pypi/l/data-syncmaster.svg

    :target: https://github.com/MobileTeleSystems/syncmaster/blob/develop/LICENSE.txt

.. |PyPI Python Version| image:: https://img.shields.io/pypi/pyversions/data-syncmaster.svg

    :target: https://badge.fury.io/py/data-syncmaster

.. |Docker image| image:: https://img.shields.io/docker/v/mtsrus/syncmaster-backend?sort=semver&label=docker

    :target: https://hub.docker.com/r/mtsrus/syncmaster-backend

.. |Documentation| image:: https://readthedocs.org/projects/syncmaster/badge/?version=stable

    :target: https://syncmaster.readthedocs.io

.. |Build Status| image:: https://github.com/MobileTeleSystems/syncmaster/workflows/Run%20All%20Tests/badge.svg

    :target: https://github.com/MobileTeleSystems/syncmaster/actions

.. |Coverage| image:: https://codecov.io/gh/MobileTeleSystems/syncmaster/graph/badge.svg?token=ky7UyUxolB

    :target: https://codecov.io/gh/MobileTeleSystems/syncmaster

.. |pre-commit.ci| image:: https://results.pre-commit.ci/badge/github/MobileTeleSystems/syncmaster/develop.svg

    :target: https://results.pre-commit.ci/latest/github/MobileTeleSystems/syncmaster/develop

What is Data.SyncMaster?

------------------------

Data.SyncMaster is as low-code ETL tool for transfering data between databases and file systems.

List of currently supported connections:

* Apache Hive

* Clickhouse

* Postgres

* Oracle

* MSSQL

* MySQL

* HDFS

* S3

Current Data.SyncMaster implementation provides following components:

* REST API

* Celery Worker

**Note**: service is under active development, and is not ready to use.

Goals

-----

* Make transfering data between databases and file systems as simple as possible

* Provide a lot of builtin connectors to transfer data in heterogeneous environment

* RBAC and multitenancy support

Non-goals

---------

* This is not a backup system

* This is not a CDC solution

* Only batch, no streaming

.. documentation

Documentation

-------------

See https://syncmaster.readthedocs.io