Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/badouralix/rclone-lambda-sync
A lambda function to run `rclone sync` periodically
https://github.com/badouralix/rclone-lambda-sync
aws aws-lambda cloud rclone storage
Last synced: about 8 hours ago
JSON representation
A lambda function to run `rclone sync` periodically
- Host: GitHub
- URL: https://github.com/badouralix/rclone-lambda-sync
- Owner: badouralix
- License: mit
- Created: 2021-05-15T22:34:07.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-11-27T17:38:40.000Z (12 months ago)
- Last Synced: 2024-05-01T14:05:13.849Z (7 months ago)
- Topics: aws, aws-lambda, cloud, rclone, storage
- Language: Python
- Homepage:
- Size: 300 KB
- Stars: 27
- Watchers: 3
- Forks: 5
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
# Rclone Lambda Sync
[](https://github.com/badouralix/rclone-lambda-sync/actions/workflows/lint.yaml)
A lambda function to run `rclone sync` periodically.
## Prerequisites
Install a lambda layer with rclone, such as [rclone-lambda-layer](https://github.com/badouralix/rclone-lambda-layer).
Create a valid rclone config with two remotes `source` and `destination`. See for details.
```bash
rclone config create "source" ...
rclone config create "destination" ...
```
This creates a local file that looks like this :
```bash
$ cat ~/.config/rclone/rclone.conf
[source]
...
[destination]
...
```
Upload this config in the SSM parameter `rclone-config`, either using the console or the following one-liner :
```bash
aws ssm put-parameter --region eu-west-3 --name rclone-config --type SecureString --value file://$HOME/.config/rclone/rclone.conf --description "Entire rclone.conf for rclone-lambda"
```
Also set `DEPLOYMENT_BUCKET` and `EVENTS_FILE` in a new `.env` local file.
## Deploy
```bash
serverless deploy
```
## Test
```bash
serverless invoke --function sync
```
## Configuration
| Environment Variable | Description | Default |
| :-------------------------------: | :---------------------------------------------------------------------------------: | :----------------------------------------------: |
| `RCLONE_CONFIG_SSM_NAME` | Name of the SSM parameter to fetch the config from. | `rclone-config` |
| `RCLONE_SYNC_CONTENT_DESTINATION` | Name of the sync destination in the format "dest:path". | `destination:/` |
| `RCLONE_SYNC_CONTENT_SOURCE` | Name of the sync source in the format "source:path". | `source:/` |
| `RCLONE_SYNC_DRY_RUN` | Do a trial run with no permanent changes. | `false` |
| `RCLONE_SYNC_EXTRA_FLAGS` | List of flags passed to rclone. See available flags in . | `--exclude /Downloads/** --exclude /External/**` |
## Schedules
By default, `rclone-lambda-sync` runs once a day around 00:00 UTC. See [rclone_lambda_daily.yaml](./rclone_lambda_daily.yaml). This behavior can be customized by following these steps :
1. Create a custom yaml file containing the desired schedules
```yaml
- schedule:
name: rclone-lambda-daily-1
rate: cron(0 1 * * ? *)
input:
RCLONE_CONFIG_SSM_NAME: rclone-config-1
- schedule:
name: rclone-lambda-daily-2
rate: cron(0 2 * * ? *)
input:
RCLONE_CONFIG_SSM_NAME: rclone-config-2
```
1. Change `EVENTS_FILE` value to the name of the custom yaml file in the `.env` local file
1. Test with a slightly different command
```bash
serverless invoke --function sync --data '{"RCLONE_CONFIG_SSM_NAME": "rclone-config-1"}'
```
## Documentation
-
-
## License
Unless explicitly stated to the contrary, all contents licensed under the [MIT License](LICENSE).