https://github.com/compose-x/s3-autosync
Automatically sync files from/to AWS S3
https://github.com/compose-x/s3-autosync
Last synced: 7 months ago
JSON representation
Automatically sync files from/to AWS S3
- Host: GitHub
- URL: https://github.com/compose-x/s3-autosync
- Owner: compose-x
- License: mpl-2.0
- Created: 2021-12-30T16:01:01.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2023-03-02T08:26:05.000Z (over 3 years ago)
- Last Synced: 2025-06-05T00:45:56.929Z (12 months ago)
- Language: Python
- Size: 216 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.rst
- Changelog: HISTORY.rst
- Contributing: CONTRIBUTING.rst
- License: LICENSE
Awesome Lists containing this project
README
=======================
aws-s3-files-autosync
=======================
Files watcher program to autosync with AWS S3
Inspiration
================
Failed to find libraries that work properly to deal with AWS S3 files in the same way *watchdog* does for local
filesystem changes.
The files changed in S3 are pulled down and the local changes are uploaded to S3 **with a backup in S3 prior to upload**
(avoids accidental loss of files).
Usage
======
.. code-block:: bash
files_s3_autosync -h
usage: files_s3_autosync [-h] [-f FILE_PATH | -e ENV_VAR] [_ ...]
positional arguments:
_
optional arguments:
-h, --help show this help message and exit
-f FILE_PATH, --from-file FILE_PATH
Configuration for execution from a file
-e ENV_VAR, --from-env-var ENV_VAR
Configuration for execution is in an environment variable
Input files model
.. literalinclude:: aws_s3_files_autosync/input.json
:language: json
Features
--------
* Synchronize (GET/PUT) files from/to local filesystem to S3.
* Validates whether downloading the file is necessary based on file size and timestamps
* Automatically creates a copy of the current object in S3 before uploading a newer version.
* Simulates **aws s3 sync** for specific files