https://github.com/aws-samples/cloud-storage-to-s3-tool

https://github.com/aws-samples/cloud-storage-to-s3-tool

Last synced: 5 days ago
JSON representation

• Host: GitHub
• URL: https://github.com/aws-samples/cloud-storage-to-s3-tool
• Owner: aws-samples
• License: mit-0
• Created: 2023-05-08T02:17:07.000Z (over 1 year ago)
• Default Branch: main
• Last Pushed: 2024-06-18T03:18:53.000Z (5 months ago)
• Last Synced: 2024-07-11T15:49:02.575Z (4 months ago)
• Language: Java
• Size: 3.32 MB
• Stars: 4
• Watchers: 7
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • Contributing: CONTRIBUTING.md
  • License: LICENSE
  • Code of conduct: CODE_OF_CONDUCT.md

Awesome Lists containing this project

README

        
Cloud Storage To AWS S3 Tool

======================================

This tool is used to transfer data from Google GCS and other cloud providers (supported by Rclone) to AWS S3. It allows you to batch start and manage EC2 instances to transfer a large amount of data in parallel.

The architecture diagram based on Google GCS as the data source is as follows:

![architecture.png](architecture.png)

Component functions:

- Amazon SQS: Stores and sorts transfer tasks after they are issued.

- Amazon DynamoDB: Automatically creates three tables for storing transfer devices, task status, and configurations during system initialization.

Prerequisites

-------------

Building the project

--------------------

Navigate to the newly created folder:

Run the project with:

    mvn clean spring-boot:run

Navigate to:

    http://localhost:8080

Login with: `user` and `demo`

Package the application

-----------------------

To package the project run:

    mvn clean package

We need to use the packaged jar file as the executable program.

The deployment document can be found in the deployment-guidance.pdf file in the repository. The document is in Chinese. If you need an English version, please feel free to contact us.

## Security

See [CONTRIBUTING](CONTRIBUTING.md#security-issue-notifications) for more information.

## License

This library is licensed under the MIT-0 License. See the LICENSE file.