{"id":13713909,"url":"https://github.com/maknahar/s3copy","last_synced_at":"2026-01-17T15:33:07.868Z","repository":{"id":207175074,"uuid":"125908788","full_name":"maknahar/s3copy","owner":"maknahar","description":"An AWS lambda function to duplicate objects from n to m AWS S3 Buckets where n, m \u003e= 1","archived":false,"fork":false,"pushed_at":"2018-04-10T17:17:03.000Z","size":8557,"stargazers_count":1,"open_issues_count":0,"forks_count":3,"subscribers_count":1,"default_branch":"master","last_synced_at":"2024-08-03T23:28:58.340Z","etag":null,"topics":["amazon-web-services","aws-lambda","aws-s3","copy","duplicate-files","duplication","go","golang","lambda"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maknahar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2018-03-19T19:20:42.000Z","updated_at":"2022-11-14T10:01:11.000Z","dependencies_parsed_at":"2023-11-14T13:50:35.354Z","dependency_job_id":null,"html_url":"https://github.com/maknahar/s3copy","commit_stats":null,"previous_names":["maknahar/s3copy"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maknahar%2Fs3copy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maknahar%2Fs3copy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maknahar%2Fs3copy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maknahar%2Fs3copy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maknahar","download_url":"https://codeload.github.com/maknahar/s3copy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224551100,"owners_count":17330070,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-web-services","aws-lambda","aws-s3","copy","duplicate-files","duplication","go","golang","lambda"],"created_at":"2024-08-02T23:01:47.403Z","updated_at":"2026-01-17T15:33:07.854Z","avatar_url":"https://github.com/maknahar.png","language":"Go","funding_links":[],"categories":["Repositories"],"sub_categories":[],"readme":"# s3copy\nA lambda function to copy objects from n to m S3 Buckets\n where n, m \u003e= 1\n \n \n## Deployment\n- Download the deployment file from \nhttps://github.com/maknahar/s3copy/releases\n- Upload the deployment file to lambda\n- Set the configuration\n\nor\n\n- Checkout the latest release code \n`go get github.com/maknahar/copys3`\n- Build and generate the deployment file \n`GOOS=linux go build -o main main.go \u0026\u0026 zip deployment.zip main`\n- Upload the deployment file to lambda\n- Set the configuration \n\n**Handler function name:** `main`\n\n## Config\nThis Lambda Function expect the list of source and \ndestination buckets in a JSON file.\n\n\n### Config File Format  \n\nConfig File is a map of input s3 bucket and detail to copy \nchanges to destination.\n\nThe function can take events from an SQS as well if \nsource bucket put all event to an SQS.\n\nThe function can take events from an SQS where SQS is \nsubscribed to an SNS and source bucket publish a message \nto an SNS on every change.\n\nTo Support SQS processing add SQS URL `sqs` and\nSQS region `sqsRegion` in the config file. Otherwise,\nLeave them empty `\"\"` or remove the keys.\n\n```\n{\n  \"source-bucket\": {\n    \"region\": \"us-east-1\",\n    \"sqs\": \"URL of SQS where all S3 change events are stored\",\n    \"sqsRegion\": \"us-east-1\",\n    \"destinations\": [\n      \"destination-bucket-1\",\n      \"destination-bucket-2\"\n    ]\n  },\n  \"another-source-bucket\": {\n    \"region\": \"us-east-1\",\n    \"sqs\": \"SQS URL\",\n    \"sqsRegion\": \"us-east-1\",\n    \"destinations\": [\n      \"destination-bucket-1\",\n      \"destination-bucket-3\"\n    ]\n  }\n}\n```\n\nWith above configuration, any object put in `source-bucket` \nwill be copied to `destination-bucket-1` \u0026 `destination-bucket-2`\nand any object put in `another-source-bucket` will be \ncopied to `destination-bucket-1` \u0026 `destination-bucket-3`.\n\n\n### Env Var\nThis Lambda function can be configured in two ways. \n- Either give a public url of config file via CONFIG_FILE\n- or provide base64 encoded value of configuration via \nCONFIG. You can online tools like www.base64encode.org\nto encode config content.\n\nIf both are provided CONFIG_FILE value will take precedence.\n\n- CONFIG_FILE : URL of configuration file\n\n- CONFIG : Base64 Encoded string of content of CONFIG_FILE\n\n## NOTE:\nMake sure Lambda function have required access to source \nand destination bucket. \n\n## Contribution\nAll contributions are welcome. Either via PR or Issue.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaknahar%2Fs3copy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaknahar%2Fs3copy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaknahar%2Fs3copy/lists"}