{"id":39811411,"url":"https://github.com/yandex-cloud-examples/yc-s3-data-sync-to-aws","last_synced_at":"2026-01-18T12:44:35.540Z","repository":{"id":297614751,"uuid":"769055776","full_name":"yandex-cloud-examples/yc-s3-data-sync-to-aws","owner":"yandex-cloud-examples","description":"Синхронизация данных между S3 хранилищами в Yandex Cloud и AWS.","archived":false,"fork":false,"pushed_at":"2025-06-06T11:44:22.000Z","size":67,"stargazers_count":1,"open_issues_count":0,"forks_count":1,"subscribers_count":10,"default_branch":"main","last_synced_at":"2025-06-06T12:38:29.965Z","etag":null,"topics":["aws","aws-lambda","data-sync","object-storage","s3","s3-storage","serverless","terraform","yandex-cloud","yandex-function","yandexcloud"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yandex-cloud-examples.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-03-08T08:53:15.000Z","updated_at":"2025-06-06T11:44:21.000Z","dependencies_parsed_at":"2025-06-06T12:38:38.339Z","dependency_job_id":"41df6822-d7c7-4e5d-bfa8-7f109efa15a2","html_url":"https://github.com/yandex-cloud-examples/yc-s3-data-sync-to-aws","commit_stats":null,"previous_names":["yandex-cloud-examples/yc-s3-data-sync-to-aws"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/yandex-cloud-examples/yc-s3-data-sync-to-aws","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-cloud-examples%2Fyc-s3-data-sync-to-aws","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-cloud-examples%2Fyc-s3-data-sync-to-aws/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-cloud-examples%2Fyc-s3-data-sync-to-aws/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-cloud-examples%2Fyc-s3-data-sync-to-aws/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yandex-cloud-examples","download_url":"https://codeload.github.com/yandex-cloud-examples/yc-s3-data-sync-to-aws/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yandex-cloud-examples%2Fyc-s3-data-sync-to-aws/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28536012,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T10:13:46.436Z","status":"ssl_error","status_checked_at":"2026-01-18T10:13:11.045Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-lambda","data-sync","object-storage","s3","s3-storage","serverless","terraform","yandex-cloud","yandex-function","yandexcloud"],"created_at":"2026-01-18T12:44:35.448Z","updated_at":"2026-01-18T12:44:35.515Z","avatar_url":"https://github.com/yandex-cloud-examples.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Description\n\nThis scenario explains how to leverage [AWS Lambda](https://aws.amazon.com/lambda/) functions and [Yandex Cloud Functions](https://cloud.yandex.com/en-ru/docs/functions/) to sync newly created objects on [AWS S3](https://aws.amazon.com/s3/) and [Yandex Object Storage](https://cloud.yandex.com/en-ru/docs/storage/).\n\nThe example uses the same function code for both sides, as Yandex Cloud Functions and AWS Lambda runtimes are compatible.\n\nIf you look at the code, you’ll notice that it uses the same S3 API for both sides, as the AWS S3 and Yandex Storage APIs are compatible.\n\n\n\u003cp align=\"center\"\u003e\n    \u003cimg src=\"s3-sync.png\" alt=\"S3 Synchronization diagram\" width=\"800\"/\u003e\n\u003c/p\u003e\n\n# Limitations\n\nPlease note that this example syncs only newly created objects. If you want some solution that helps with more scenarios out of the box please refer to \u003ca href=\"../Multi-cloud S3 storage/README.md\"\u003eMulti-cloud storage with Yandex Cloud and Amazon S3 guide\u003c/a\u003e.\n\n\n# Syncing two S3 buckets\n\nReview the following example to see how syncing works.\nYou will need:\n\n- Accounts in AWS and Yandex Cloud\n- Bash\n- Terraform 1.1.5\n- [s3cmd](https://s3tools.org/download)\n- jq\n\nConfigure the AWS site:\n- Configure the [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)\n\n\nConfigure Yandex Cloud:\n- Configure the [YC CLI](https://cloud.yandex.com/docs/cli/quickstart) \n- Prepare the Terraform environment:\n\n```\nexport YC_TOKEN=$(yc iam create-token)\nexport YC_CLOUD_ID=$(yc config get cloud-id)\nexport YC_FOLDER_ID=$(yc config get folder-id)\nexport TF_VAR_folder_id=$YC_FOLDER_ID\n```\n\n\n## Quick start\n\n### Initiate an example playbook  \n\n```\ncd example\nterraform init\nterraform apply\n```\n\n### Check results\n\nWe will use:\n- `aws` tool for AWS site\n- `s3cmd` tool for Yandex Cloud site\n\nPrepare an environment:\n```\nBUCKET_NAME=$(terraform output -raw bucket_name)\nmv s3cfg ~/.s3cfg\n```\n\nMake sure that AWS and Yandex Cloud buckets are empty:\n```\naws s3 ls $BUCKET_NAME\ns3cmd ls s3://$BUCKET_NAME\n```\n\nPut the file into the Yandex Cloud bucket:\n```\ns3cmd put sync.zip s3://$BUCKET_NAME \n\nupload: 'sync.zip' -\u003e 's3://yc-s3-sync-a6v4g3vlra/sync.zip'  [1 of 1]\n 2551 of 2551   100% in    0s    24.73 KB/s  done\n```\n\nCheck that the file is in place at Yandex Cloud:\n```\ns3cmd ls s3://$BUCKET_NAME\n\n2020-12-08 18:41         2551  s3://yc-s3-sync-a6v4g3vlra/sync.zip\n```\n\nCheck that the file is successfully synced to the AWS S3:\n```\naws s3 ls $BUCKET_NAME\n\n2020-12-08 21:42:14       2551 sync.zip\n```\n\n### To destroy everything quickly\n\nDelete all files at AWS and Yandex Cloud buckets:\n```\naws s3 rm s3://$BUCKET_NAME/sync.zip\ns3cmd rm s3://$BUCKET_NAME/sync.zip\nrm ~/.s3cfg\n```\n\nDestroy Terraform resources:\n```\nterraform destroy\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyandex-cloud-examples%2Fyc-s3-data-sync-to-aws","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyandex-cloud-examples%2Fyc-s3-data-sync-to-aws","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyandex-cloud-examples%2Fyc-s3-data-sync-to-aws/lists"}