https://github.com/yassineazzouz/tanit

Kraken - A distributed data transfer tool.
https://github.com/yassineazzouz/tanit

aws-s3 data-backup data-life-cycle data-migration data-transfer disaster-recovery gcs hdfs python thrift

Last synced: 23 days ago
JSON representation

Kraken - A distributed data transfer tool.

• Host: GitHub
• URL: https://github.com/yassineazzouz/tanit
• Owner: yassineazzouz
• License: mit
• Created: 2020-08-30T04:17:38.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2020-11-08T23:05:13.000Z (over 5 years ago)
• Last Synced: 2025-09-29T19:10:44.913Z (7 months ago)
• Topics: aws-s3, data-backup, data-life-cycle, data-migration, data-transfer, disaster-recovery, gcs, hdfs, python, thrift
• Language: Python
• Homepage:
• Size: 500 KB
• Stars: 2
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.rst
  • License: LICENSE

Awesome Lists containing this project

README

          
.. image:: https://img.shields.io/pypi/v/tanit

    :target: https://pypi.org/project/tanit

.. image:: https://img.shields.io/pypi/pyversions/tanit

    :target: https://pypi.org/project/tanit

.. image:: https://img.shields.io/travis/com/yassineazzouz/tanit

    :target: https://travis-ci.com/yassineazzouz/tanit

.. image:: https://img.shields.io/codecov/c/github/yassineazzouz/tanit

    :target: https://codecov.io/gh/yassineazzouz/tanit

Tanit

==================================

Tanit is a distributed, fast, and reliable data transfer service for efficiently moving large amounts of data between different storage technologies. It uses a distributed master worker architecture, to allow very fast data transfer speed and horizontal scaling, while implementing multiple failover, recovery and data consistency checks mechanisms to ensure robustness, fault tolerance and reliability.

*Tanit* typical use cases include but are not limited to :

- Data migration (across storage technologies)

- Data Replication.

- Backup and Disaster recovery.

- Data life cycle management.

Features

--------

* Distributed data transfer service.

* Support multiple storage technologies.

* Easy to setup, configurable services.

* Command line interface to submit and monitor jobs.

* Support multiple scheduling algorithms and job priority.

* Support multiple jobs placement policies across workers.

Install

---------------

::

    pip install tanit-pyds