https://github.com/sloppycoder/s3-utils
https://github.com/sloppycoder/s3-utils
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/sloppycoder/s3-utils
- Owner: sloppycoder
- Created: 2024-01-11T10:48:51.000Z (over 2 years ago)
- Default Branch: develop
- Last Pushed: 2024-03-18T15:39:55.000Z (about 2 years ago)
- Last Synced: 2025-01-11T08:14:01.397Z (over 1 year ago)
- Language: Python
- Size: 32.2 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Utilites for working with AWS S3 objects
```shell
# extract data from S3 objects and store them into a JSON file for later use
python get_file_list.py --prefix="video/" --bucket=brick.vino9.net --output=video_file_list.json
# compare the file listed in input file and check if the corresponding local file exists and is of the same size
python check_files.py --input=video_file_list.json --base=/volume1/media
# using bulk delete API to delete objects from S3 bucket quickly
python bulk_del.py --bucket=brick.vino9.net --prefix=blackhole/old
```
## AWS CLI for restore files from Glacier
```shell
# get a list of files
aws s3api list-objects --bucket bucket_name_ --query 'Contents[].Key' --prefix="prefix/" | jq -r '.[]' > obj_list.txt
# check the restore status for objects in a list, initiate restore if it is yet to start
./restore_status < obj_list.txt
```