Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cloud303-cholden/rs-s3-sync
Manually sync system files and Amazon S3.
https://github.com/cloud303-cholden/rs-s3-sync
file-upload rust s3 sync
Last synced: 1 day ago
JSON representation
Manually sync system files and Amazon S3.
- Host: GitHub
- URL: https://github.com/cloud303-cholden/rs-s3-sync
- Owner: cloud303-cholden
- License: bsd-3-clause
- Created: 2023-03-21T23:45:49.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-04-22T19:43:06.000Z (10 months ago)
- Last Synced: 2025-01-28T07:02:01.044Z (9 days ago)
- Topics: file-upload, rust, s3, sync
- Language: Rust
- Homepage:
- Size: 40 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# S3 Sync
### Features
- Sync files to and from a single S3 bucket.
- Configure multiple directories to be synced to S3.
- Currently, the functionality is only provided via running the binary manually. This works for my use case, but is definitely not ideal.
### Installation
Currently, only installation with `cargo` is supported. This will install the executable to `$HOME/.cargo/bin/s3-sync`.
```sh
cargo install --git https://github.com/cloud303-cholden/rs-s3-sync
```
### Example Configuration
The TOML configuration location must either be `$S3_SYNC_CONFIG` or `$XDG_CONFIG_HOME/s3-sync/config.toml`. Below is a sample configuration file.
```toml
# Muliple paths are supported.
paths = [
"/home/user/Documents/dir2",
"/home/user/Documents/dir2",
]
[aws]
bucket = "aws-bucket-name" # Only the name is needed. No S3 URI prefix required.
profile = "aws-profile" # AWS profile name. If not specified, AWS credentials will be retrieved from the environment.
```
### Usage
This is just a simple executable for now. No arguments or flags are supported.
```sh
s3-sync
```
### Implementation Notes
- Currently the full path of system files are copied to S3. This means that syncing files between systems will only work as expected if the paths are the exact same in both systems.
- The logic for deciding between uploading versus downloading is the last modified date of the file. This is tracked in a `manifest.json` file in the root of the S3 bucket. Deleting or altering this manifest will produce unexpected results.
### Future
- The decision to either download or upload is done by comparing the last modified date of a file both locally and in S3. This is very quick to check, but a more robust implementation might also compare the file hashes.
- Need logic for handling deletion of files both locally and in S3. One example scenario is whether deleting a file locally should prevent it from being downloaded when the next sync is run.
- Add difference calculation.
- Add a service for automatically watching for file changes. Move core logic to library.
- Change the binary to an actual CLI.
- Add support for changing configuration settings with CLI.