{"id":20697504,"url":"https://github.com/epomatti/aws-data-integrity","last_synced_at":"2026-04-25T08:34:55.970Z","repository":{"id":189591609,"uuid":"680913248","full_name":"epomatti/aws-data-integrity","owner":"epomatti","description":"Data lock mechanisms available on AWS.","archived":false,"fork":false,"pushed_at":"2023-08-21T23:10:55.000Z","size":231,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-17T18:36:36.906Z","etag":null,"topics":["aws","aws-backup","aws-glacier","aws-security","backup","glacier","lock","mfa","s3","terraform","vault","worm"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epomatti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-20T20:00:34.000Z","updated_at":"2024-01-02T02:25:21.000Z","dependencies_parsed_at":"2023-08-20T22:35:11.902Z","dependency_job_id":"eacf2f97-fec1-47b5-829b-790be9c1f471","html_url":"https://github.com/epomatti/aws-data-integrity","commit_stats":null,"previous_names":["epomatti/aws-data-integrity-options"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Faws-data-integrity","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Faws-data-integrity/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Faws-data-integrity/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epomatti%2Faws-data-integrity/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epomatti","download_url":"https://codeload.github.com/epomatti/aws-data-integrity/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242961747,"owners_count":20213315,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-backup","aws-glacier","aws-security","backup","glacier","lock","mfa","s3","terraform","vault","worm"],"created_at":"2024-11-17T00:18:37.168Z","updated_at":"2026-04-25T08:34:55.965Z","avatar_url":"https://github.com/epomatti.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# AWS Data Integrity\n\nSet of Data Integrity features provided by AWS services as a requirement for AWS Security.\n\n## Scope\n\n### Locks and Hold\n\nThe following services provide data integrity features:\n\n- [S3 Object Lock](#s3-object-lock)\n- [S3 Glacier Vault Lock](#s3-glacier-vault-lock)\n- [AWS Backup Vault Lock](#aws-backup-vault-lock)\n\nAdditionally, Legal Hold is available for S3 and Backup Vault.\n\nLegal holds can be applied to:\n- S3 Object lock legal hold (requires S3 object lock to be enabled). Object-level setting.\n- AWS Backup legal hold (does not require vault lock). Prevents backups from being deleted for duration of lock. All or selected backups.\n\n### EC2 Snapshots\n\nData lifecycle management (DLM) for EC2/EBS.\n\n## Pre-requisite: Infrastructure Setup\n\nCopy the sample variables file:\n\n```sh\ncp config/sample.tfvars .auto.tfvars\n```\n\nToggle which module to create:\n\n```terraform\ntoggle_create_s3      = true\ntoggle_create_glacier = false\ntoggle_create_backup  = false\n```\n\nTo create the baseline infrastructure, apply the configuration:\n\n```sh\nterraform init\nterraform apply\n```\n\n## S3 Object Lock\n\nKey properties of this feature:\n\n- Objects are **immutable** during a retention period.\n- Can apply only for NEW buckets on creation. Not possible to apply this for existing buckets.\n- Requires and automatically enables versioning, and it is not possible to pause or disable it either.\n- New objects inherit default bucket settings, if defined. Retention period is applied on an object-by-object basis.\n\nThere are two retention period modes:\n\n- **Governance** - Objects are immutable. However, authorization is granted via the `s3:BypassGovernanceRetention` action.\n- **Compliance** - No one is allowed to change the object until the lock has expired. Not even the AWS Account Root user.\n\nS3 Object Legal Hold prevents objects from being **deleted or overwritten** while active.\n\n### Retention period type `GOVERNANCE`\n\nWhile no retention period is defined, it is possible to apply changes to the objects.\n\nUpload a new version `v2` of the file:\n\n```sh\naws s3api put-object --bucket bucketdataintegritysandbox789 --key \"important.txt\" --body \"artifacts/important-v2.txt\"\n```\n\nNow, apply an object retention period for the object:\n\nℹ️ Modify the value for `RetainUntilDate` accordingly.\n\n```sh\naws s3api put-object-retention \\\n    --bucket bucketdataintegritysandbox789 \\\n    --key \"important.txt\" \\\n    --retention '{ \"Mode\": \"GOVERNANCE\", \"RetainUntilDate\": \"2023-08-20T20:40:00-03:00\" }'\n```\n\nWith a test user that does not have `s3:BypassGovernanceRetention`, try again to create a new object `v3` version:\n\n```sh\naws s3api put-object --bucket bucketdataintegritysandbox789 --key \"important.txt\" --body \"artifacts/important-v3.txt\"\n```\n\nYou should expect a failure at this point. Only users with `s3:BypassGovernanceRetention` can perform this operation or delete the object.\n\n\n### Retention period type `COMPLIANCE`\n\nThis section will use the same object key `important.txt`, so first it is necessary to remove the retention:\n\n\u003e 💡 Using `--bypass-governance-retention` toggle to allow this operation\n\n```\naws s3api put-object-retention \\\n    --bucket bucketdataintegritysandbox789 \\\n    --key \"important.txt\" \\\n    --retention '{ }' \\\n    --bypass-governance-retention\n```\n\n\u003e ℹ️ Setting the `RetainUntilDate` value must be done with attention. It's not possible revert this action until the retention period is done.\n\n```\naws s3api put-object-retention \\\n    --bucket bucketdataintegritysandbox789 \\\n    --key \"important.txt\" \\\n    --retention '{ \"Mode\": \"COMPLIANCE\", \"RetainUntilDate\": \"2023-08-20T20:40:00-03:00\" }'\n```\n\nIt is **NOT** possible to disable `COMPLIANCE` retention periods. This should return `Access Denied`:\n\n```\naws s3api put-object-retention \\\n    --bucket bucketdataintegritysandbox789 \\\n    --key \"important.txt\" \\\n    --retention '{ }' \\\n    --bypass-governance-retention\n```\n\n### Legal Hold ###\n\nLegal holds will prevent the objects from being **deleted or overwritten**:\n\n```sh\naws s3api put-object-legal-hold \\\n    --bucket bucketdataintegritysandbox789 \\\n    --key \"important.txt\" \\\n    --legal-hold Status=ON\n```\n\nDeleting this object version with Legal Hold is not permitted.\n\n## S3 Glacier Vault Lock\n\n- Operates with a Resource Policy.\n- Denies anyone the `DeleteArchive` unless conditions are met.\n- Used in addition to IAM or vault access policies.\n\nWhen a lock is applied it stays for \"In-progress\" state for 24 hours. It is possible to teste everything during this period.\n\n🚨 Vault lock policy cannot be modified or deleted after confirmation.\n\nAs shown in the print bellow, Terraform will create the Glacier Vault along with the Vault Lock policy.\n\n\u003e ℹ️ Created with `complete_lock = false` so it will automatically expire after 24h.\n\n\u003cimg src=\".assets/glacier.png\" /\u003e\n\nTo test it, upload an archive:\n\n```sh\naws glacier upload-archive --account-id ... --vault-name dataintegritysandboxglacier789 --body \"artifacts/archive.txt\"\n```\n\nNow try deleting the archive:\n\n```sh\naws glacier delete-archive \\\n    --account-id ...  \\\n    --vault-name dataintegritysandboxglacier789 \\\n    --archive-id ...\n```\n\nAn `explicit deny` error should be returned.\n\nAfter deleting the vault lock policy, trying again should allow the operation to go through.\n\nCheck the [documentation][1] for Glacier vault lock policies.\n\n## AWS Backup Vault Lock\n\nAWS Backup Vault Locks will prevents backups in the vault from being deleted until lock expiration.\n\nThere is a difference between these retention periods:\n\n- **Backup Job retention period** - Defines how long a backup job should be retained.\n- **Backup Vault Lock retention period** - Retention period backup job.\n\nEnabling the vault lock will also protect the Backup Vault from being deleted indefinitely, unlike Glacier.\n\nSame as with S3 locks, there are retention modes:\n\n- **Governance** - Users may be authorized to delete objects or vault with special grants.\n- **Compliance** - Enables with start date (at least 3 days in the future). During this grace period the lock may be modified or removed. After this, the vault is immutable, and no one can delete backups or delete/manage the vault, forever. Only terminating the AWS account.\n\n### Vault Lock hands-on\n\nTerraform will create a DynamoDB and a backup for it, with a vault lock already running in grace time:\n\n\u003cimg src=\".assets/backup-grace-time.png\" /\u003e\n\nStart an on-demand backup job for the DynamoDB table:\n\n```sh\naws backup start-backup-job \\\n    --backup-vault-name \"DynamoDBVault\" \\\n    --resource-arn \"dynamo-arn\" \\\n    --iam-role-arn \"role-arn\"\n```\n\nWhile locks are active, it is not possible to delete recovery points:\n\n\u003cimg src=\".assets/recoverypoint-delete.png\" /\u003e\n\n\n### Legal Hold\n\nAdditionally, it is possible to apply a Legal Hold:\n\n\u003cimg src=\".assets/backup-legalhold.png\" /\u003e\n\n[1]: https://docs.aws.amazon.com/amazonglacier/latest/dev/vault-lock-policy.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepomatti%2Faws-data-integrity","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepomatti%2Faws-data-integrity","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepomatti%2Faws-data-integrity/lists"}