https://github.com/fredlackey/lambda-s3-copy-example-node
Simple example function showing how to copy files between S3 buckets using AWS Lambda.
https://github.com/fredlackey/lambda-s3-copy-example-node
aws aws-lambda aws-s3 aws-s3-javascript lambda-function lambda-functions nodejs s3 s3-bucket s3-buckets s3-storage
Last synced: 4 months ago
JSON representation
Simple example function showing how to copy files between S3 buckets using AWS Lambda.
- Host: GitHub
- URL: https://github.com/fredlackey/lambda-s3-copy-example-node
- Owner: FredLackey
- License: apache-2.0
- Created: 2020-07-09T01:19:34.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2020-07-09T21:20:34.000Z (over 5 years ago)
- Last Synced: 2025-03-18T03:23:04.299Z (9 months ago)
- Topics: aws, aws-lambda, aws-s3, aws-s3-javascript, lambda-function, lambda-functions, nodejs, s3, s3-bucket, s3-buckets, s3-storage
- Language: JavaScript
- Homepage:
- Size: 19.5 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Copy Objects Between S3 Buckets Using AWS Lambda & NodeJS
Simple example function showing how to copy files between S3 buckets using AWS Lambda.
## Background
A friend needed a way to copy all S3 objects created by the Simple Email Service (SES) and copy them to a date-organized structure within a different S3 bucket.
## Variables
Three environment variables exist:
**`QUANTITY`**
Optional numeric to force the `getObjects` operation to limit the number of returned objects. If not set, default from S3 is 1,000 objects. NOT setting this may cause timeouts.
**`SOURCE_BUCKET`**
Name of the bucket to copy objects from.
**`TARGET_BUCKET`**
Name of the bucket to copy objects to.
## Process
The steps are as follows:
1. Get a list of all objects
2. Loop through objects
3. Check if target object exists
4. Copy object to target if new
## Recommendation
If you use this, I recommend adding a `deleteObject` from the `SOURCE_BUCKET` location to speed up additional processing later on. Also, since no _starting_ position is possible, it is likely you will always iterate over the same 1,000 objects.
## Contact Info
As always, if you find this and need a hand, feel free to reach out:
**Fred Lackey**
**fred.lackey@gmail.com**
**http://fredlackey.com**