https://github.com/pcpratheesh/s3fileszipper
Amazon lambda function to read s3 files and create as zip
https://github.com/pcpratheesh/s3fileszipper
Last synced: about 2 months ago
JSON representation
Amazon lambda function to read s3 files and create as zip
- Host: GitHub
- URL: https://github.com/pcpratheesh/s3fileszipper
- Owner: pcpratheesh
- Created: 2020-09-27T04:47:34.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-09-28T04:25:51.000Z (almost 6 years ago)
- Last Synced: 2025-10-06T22:55:00.796Z (9 months ago)
- Language: Go
- Size: 13.5 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Security: SECURITY.md
Awesome Lists containing this project
README
#
GoLang (google go)
# s3FilesZipper
Amazon lambda golang function to read s3 bucket files and create as a zip.
Move the created zip file into same or another bucket destination.
# Steps
- Load Configuration
- Read files from s3 related with the path provided in configuration
- Create zip
- Copy zip into destintion s3 bucket path
# Deployment
- Download the deployment file from https://github.com/pratheeshpcplpta/s3FilesZipper
- Build the go with the following command - **env GOOS=linux go build -o main main.go && zip deployment.zip main**
- Upload the deployment file to lambda
- Set the configuration
Handler function name: main
# Configurations
You have to provide the configuration as base64 encoded format. Either add **CONFIG** in lambda variable configuration or you can pass the **config** as parameter if the lambda trigger event is like amzon API interface.
Use https://www.base64encode.org/ to encode the configurations
# Config Params
```json
{
"region" : "Region",
"bucket" : "Source Bucket",
"destinationbucket" : "Destination Bucket",
"filesources" : ["list of file sources"],
"tmpfilepath" : "temporary zip create path, provide if using VPC and EFS in lambda,default config will be /tmp",
"zipdestinationfilepath" : "Destination path for copy the ziped file within the destination buket"
}
```
# Important
- If you are trying to zip files larger than 512 MB size, then you have to setup VPC and EFS (Elastic File System) in lambda function. Lambda only allows 512MB size to create a tmp file.