{"id":37020726,"url":"https://github.com/adobe/shredder","last_synced_at":"2026-01-14T02:25:19.762Z","repository":{"id":52615543,"uuid":"134743415","full_name":"adobe/shredder","owner":"adobe","description":"Shredder is a Java daemon service that can perform graceful shutdowns for AWS EC2 instances in an Auto Scaling Group (ASG)","archived":false,"fork":false,"pushed_at":"2025-12-21T00:14:52.000Z","size":228,"stargazers_count":17,"open_issues_count":7,"forks_count":8,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-12-22T21:59:49.533Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adobe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-05-24T16:44:20.000Z","updated_at":"2025-09-26T16:53:41.000Z","dependencies_parsed_at":"2025-10-04T22:10:06.738Z","dependency_job_id":"d0ee36a2-5b53-41dd-bd02-b1d2e5265f6a","html_url":"https://github.com/adobe/shredder","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/adobe/shredder","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fshredder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fshredder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fshredder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fshredder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adobe","download_url":"https://codeload.github.com/adobe/shredder/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adobe%2Fshredder/sbom","scorecard":{"id":167340,"data":{"date":"2025-08-11","repo":{"name":"github.com/adobe/shredder","commit":"78d95174891e35c4481c079f07129917230e6218"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.7,"checks":[{"name":"Code-Review","score":3,"reason":"Found 6/19 approved changesets -- score normalized to 3","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Binary-Artifacts","score":9,"reason":"binaries present in source code","details":["Warn: binary detected: gradle/wrapper/gradle-wrapper.jar:1"],"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact 2.1.2 not signed: https://api.github.com/repos/adobe/shredder/releases/41892236","Warn: release artifact 2.1.1 not signed: https://api.github.com/repos/adobe/shredder/releases/41717364","Warn: release artifact 2.1.0 not signed: https://api.github.com/repos/adobe/shredder/releases/29557103","Warn: release artifact 2.0.2 not signed: https://api.github.com/repos/adobe/shredder/releases/15587551","Warn: release artifact 2.0.1 not signed: https://api.github.com/repos/adobe/shredder/releases/15583860","Warn: release artifact 2.1.2 does not have provenance: https://api.github.com/repos/adobe/shredder/releases/41892236","Warn: release artifact 2.1.1 does not have provenance: https://api.github.com/repos/adobe/shredder/releases/41717364","Warn: release artifact 2.1.0 does not have provenance: https://api.github.com/repos/adobe/shredder/releases/29557103","Warn: release artifact 2.0.2 does not have provenance: https://api.github.com/repos/adobe/shredder/releases/15587551","Warn: release artifact 2.0.1 does not have provenance: https://api.github.com/repos/adobe/shredder/releases/15583860"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found linked content: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: github.com/adobe/.github/.github/SECURITY.md:1","Info: Found text in security policy: github.com/adobe/.github/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 10 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}}]},"last_synced_at":"2025-08-16T15:16:41.154Z","repository_id":52615543,"created_at":"2025-08-16T15:16:41.154Z","updated_at":"2025-08-16T15:16:41.154Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28408711,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-14T01:52:23.358Z","status":"online","status_checked_at":"2026-01-14T02:00:06.678Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-14T02:25:18.964Z","updated_at":"2026-01-14T02:25:19.746Z","avatar_url":"https://github.com/adobe.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![Build Status](https://travis-ci.org/adobe/shredder.svg?branch=master)](https://travis-ci.org/adobe/shredder)\n\n# Shredder for EC2\n\nShredder for EC2 is a Java daemon service that can perform graceful shutdowns for AWS EC2 instances in an Auto Scaling Group (ASG).\n\nIt runs on the EC2 instance and constantly listens for a TERMINATION signal from the Auto Scaling Group (ASG). When such a signal is received, it will start running a list of defined commands (e.g. push data to S3, stop the running services gracefully). After the list of Bash commands is executed, Shredder for EC2 will send a COMPLETE lifecycle message to the Auto Scale Group (ASG). This will trigger the EC2 instance termination.\n\nThis tool is useful for services that are stateful - some data needs to be pushed before terminating the EC2 instance. Therefore, it prevents the data loss.\n\n# Install\n\nYou can either build the latest packages (see below) or you can use the published artifacts.\nFor RedHat/Centos\n```\nsudo yum install https://github.com/adobe/shredder/releases/download/2.0.2/aam-shredder-ec2-2.0.2-20190215161148.noarch.rpm\n```\n\nBy default, whenever shredder receives a shutdown signal, it will run all the scripts from `/opt/shutdown-scripts` (in lexicographical order). In our case, each application RPM that requires a cleanup logic will automatically generate a shutdown script in this shutdown folder. For instance:\n```bash\n$ tree /opt/shutdown-scripts\n/opt/shutdown-scripts\n├── myapp\n│   └── shutdown.sh\n└── route53-utils\n    └── remove_dns.sh\n\n$ cat /opt/shutdown-scripts/myapp/shutdown.sh\ns3_bucket=\"s3://aam-dcs-shutdownhook-$ENVIRONMENT\"\ns3_shutdown_data_path=\"$s3_bucket/log/$AWS_REGION/$HOSTNAME/\"\n\naws s3 cp /path/to/data/to/upload/in/s3 $s3_shutdown_data_path --recursive --exclude '*' --include '*.log' --include '*.tmp' --exclude '*processed/*'\n```\n\nNote that shredder automatically injects a set of useful environment variables when running the shutdown scripts: `$AWS_REGION`, `$AWS_ACCOUNT_ID`, `$AWS_INSTANCE_ID`, `$HOSTNAME`, `$ENVIRONMENT`.\n\nIf all the shutdown scripts exit with code 0, shredder treats this as a successful shutdown and notifies the Auto Scale Group, by sending the COMPLETE:CONTINUE lifecycle action. This will terminate the instance.\n\nHowever, if at least one of the scripts finishes with non-zero exit code, this could mean that we were not able to backup the data before terminating the instance. In this case, we would like to prevent terminating the instance, so that we can investigate what happened and recover the data (either manually or automatically). By default, if the shutdown script exits with non-zero code, shredder not terminate the instance. Instead it will wait for up to 3 days, periodically sending heartbeats to the ASG, in order to maintain the instance up. This duration can be configured via the [SHREDDER_SHUTDOWN_WAIT_TIME_IF_FAILURE](https://github.com/adobe/shredder/blob/master/shredder-ec2/src/main/resources/reference.conf#L19) environment variable. If the time expires, shredder will notify the Auto Scale Group by sending a COMPLETE:ABANDON lifecycle action and the instance will get terminated.\n\nTo find out more about Amazon EC2 Auto Scaling lifecycle hooks you can go [here](https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html).\n\n# How it works\n\n![Shredder for EC2 diagram](https://user-images.githubusercontent.com/952836/35993150-1304c044-0d15-11e8-8a2e-857c5dbbd56e.png)\n\n- When the Shredder for EC2 daemon starts, it will automatically create an SQS queue for the EC2 instance it is running on. For instance: asg-myapp-i-08d384477d1ce8643.\n- It will subscribe this SQS queue to an SNS topic described below. \n- When the Auto Scale Group decides it needs to terminate an instance, it will send a SNS notification to this topic. The SNS notification contains the instance id, that is about to be terminated (see next section on how to configure this).\n- Each Shredder-for-EC2 that is subscribed to that SNS topic will receive these nofications.\n- The Shredder-for-EC2 daemon looks at the termination messages and checks to see if the message is attributed to this instance.\n- If it is, then it will start executing the commands. After the commands are ran, Shredder for EC2 sends the COMPLETE lifecyle event to the ASG.\n\n# Configure the Auto Scale Group to send notifications to SNS when it's terminating an EC2 instance\n\nIn order for the Shredder for EC2 to be able to receive notifications when the Auto Scaling Group (ASG) wants to terminate the EC2 instance, we need to add a shutdown hook on the ASG itself.\n\nThis can be achieved with a command similar to this:\n```\naws autoscaling put-lifecycle-hook\n   --lifecycle-hook-name my-hook\n   --auto-scaling-group-name my-asg\n   --lifecycle-transition autoscaling:EC2_INSTANCE_TERMINATING\n   --role-arn arn:aws:iam::0123456789:role/myrole\n   --notification-target-arn arn:aws:sns:us-east-1:0123456789:spinnaker-shutdowns-pending'\n   --heartbeat-timeout 300\n   --default-result CONTINUE\n```\n\nReference:\n- https://docs.aws.amazon.com/autoscaling/ec2/userguide/lifecycle-hooks.html\n\n## Configure shutdown hooks in Spinnaker\n\nAlternatively, for Spinnaker (https://www.spinnaker.io), instead of running the above command manually, you can instruct the Cloud Driver to add the shutdown hook on ALL auto scale groups being created with Spinnaker.\n\n`$\u003e vim /opt/spinnaker/config/clouddriver-local.yml`\n```yaml\naws:\n  defaultAssumeRole: role/\u003c\u003e\n  accounts:\n\n    - name: \u003c\u003e-aws\n      accountId: \"\u003c\u003e\"\n      regions:\n        - name: us-east-1\n\n    - name: adev-aws\n      accountId: \"\u003c\u003e\"\n      regions:\n        - name: us-east-1\n      lifecycleHooks:\n        - defaultResult: 'CONTINUE'\n          heartbeatTimeout: 7200\n          lifecycleTransition: 'autoscaling:EC2_INSTANCE_TERMINATING'\n          notificationTargetARN: 'arn:aws:sns:{{region}}:{{accountId}}:spinnaker-shutdowns-pending'\n          roleARN: 'arn:aws:iam::{{accountId}}:role/spinnakerasg'\n```\n\n# AWS permissions\n\n## Sample IAM role \n\nThe IAM role (myrole/spinnakerasg) must have permissions to publish to the specified SNS topic:\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Effect\": \"Allow\",\n      \"Action\": [\n        \"sns:Publish\"\n      ],\n      \"Resource\": \"arn:aws:sns:us-east-1:1111111111:spinnaker-shutdowns-pending\"\n    }\n  ]\n}\n```\nThe Auto Scale Group (ASG) must be able to assume the afore mentioned role. In order to do this, we must add a Trust Relationship like this on the `myrole/spinnakerasg` role:\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Sid\": \"\",\n      \"Effect\": \"Allow\",\n      \"Principal\": {\n        \"Service\": \"autoscaling.amazonaws.com\"\n      },\n      \"Action\": \"sts:AssumeRole\"\n    }\n  ]\n}\n```\n\n\n## Sample IAM role for EC2 server running Shredder-for-EC2\n\nThe EC2 instance where you plan to deploy Shredder-for-EC2 needs have access to subscribe/unsubscribe to the SNS topic. It also needs access to the SQS queue under the desired prefix. Furthermore, the EC2 instance needs to be able to send heart beats to the Auto Scaling Group (ASG).\n\nTherefore, the IAM role used for the EC2 instance, needs to have these permissions:\n\n```json\n{\n  \"Version\": \"2012-10-17\",\n  \"Statement\": [\n    {\n      \"Action\": [\n        \"sqs:*\"\n      ],\n      \"Resource\": [\n        \"arn:aws:sqs:*:1111111111:*:*\"\n      ],\n      \"Effect\": \"Allow\"\n    },\n    {\n      \"Action\": [\n        \"sns:Unsubscribe\"\n      ],\n      \"Resource\": [\n        \"*\"\n      ],\n      \"Effect\": \"Allow\"\n    },\n    {\n      \"Action\": [\n        \"sns:Subscribe\"\n      ],\n      \"Resource\": [\n        \"arn:aws:sns:us-east-1:1111111111:*\"\n      ],\n      \"Effect\": \"Allow\"\n    },\n    {\n      \"Action\": [\n        \"autoscaling:RecordLifecycleActionHeartbeat\",\n        \"autoscaling:CompleteLifecycleAction\"\n      ],\n      \"Resource\": \"arn:aws:autoscaling:*:*:*\",\n      \"Effect\": \"Allow\"\n    }\n  ]\n}\n```\n\n# Shredder for EC2 features\n\n## Automatic variable injection\nshredder automatically injects a set of useful environment variables when running the shutdown scripts: `$AWS_REGION`, `$AWS_ACCOUNT_ID`, `$AWS_INSTANCE_ID`, `$HOSTNAME`, `$ENVIRONMENT`.\n\n## Sends heartbeats to the AWS Auto Scale group \nEven if a command takes 1 hour to run, the daemon will periodically send heartbeats to the ASG so that it keeps the EC2 instance alive\n\n# Additional cleanup on remote services\n\nAfter the EC2 instance is terminated, it is sometimes useful to notify another system about this event. For instance, you might have another service where you need to do some DNS/monitoring cleanup for the EC2 node that was terminated. If available, you could just do an HTTP call to such service, directly from the Shredder for EC2 (e.g. `https://my-remote-monitoring-service.com/remove/asg-ec2-i3`).\n\nHowever, if such an API is not available, you might want to run a daemon on the remote service itself. This daemon would listen for EC2 instance termination complete messages, and will perform the cleanup when such a message is received. This can be achieved by writing your own custom application based on the shredder-core module, which reads SNS notifications from a second topic (e.g. spinnaker-shutdown-complete). \n\nThe Shredder-for-EC2 can be instrumented to send a notification to a different SNS topic, when the cleanup is completed. You can just add a shutdown script in /opt/shutdown-scripts, which will be executed by shredder:\n\n```bash\n$ cat /opt/shutdown-scripts/notify-external.sh\n\ntarget_sns_topic=\"arn:aws:sns:$AWS_REGION:$AWS_ACCOUNT_ID:spinnaker-shutdowns-complete\"\naws sns publish --subject 'Shutdown complete' --message '{\\\"hostname\\\": \\\"$HOSTNAME\\\", \\\"app\\\": \\\"myapp\\\"}' --target-arn $target_sns_topic --region $AWS_REGION\n```\n\n# Build\n\nTo build this project:\n\n```sh\n$ git clone git@github.com:adobe/shredder.git\n$ cd shredder\n\n# Build RPM for RedHat/Centos\n$ ./gradlew clean build shredder-ec2:buildRpm\n\n# find . -name \"*rpm\"\n./shredder-ec2/build/distributions/aam-shredder-ec2-1.0.0-20180612235633.noarch.rpm\n\n# Build Debian\n$ ./gradlew clean build shredder-ec2:buildDeb\n\n# find . -name \"*deb\"\n./shredder-ec2/build/distributions/aam-shredder-ec2_1.0.0-20180612235918_noarch.deb\n```\n\n# Open in IntelliJ\n\n```sh\n$ git clone git@github.com:adobe/shredder.git\n$ cd shredder\n$ idea .\n```\n\n## Run from IntelliJ\n\nTo run shredder-ec2 locally, you can pass environment variables to bypass AWS resource lookups.\n```\nSHREDDER_CONFIG_FILE=shredder-ec2/src/main/resources/reference.conf\nAWS_PROFILE=aam-npe\nAWS_REGION=us-east-1\nAWS_REGION_ID=i-0a3806d7164d3de2f\nAWS_ACCOUNT_ID=\u003center-aam-npe-account-id\u003e\n```\n\n# Other use cases\nThe shredder-core module can be used for implementing other use cases. See the sample-cleanup sample application. \n\n# Credits\n\nProject is based on https://github.com/scopely/shudder, which is written in python.\n\n# Bugs and Feedback\n\nFor bugs, questions and discussions please use the [GitHub Issues](https://github.com/adobe/shredder/issues).\n\n\n# LICENSE\n\nCopyright 2018 Adobe Systems Incorporated\n\nLicensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at\n\nhttp://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fshredder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadobe%2Fshredder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadobe%2Fshredder/lists"}