{"id":27054082,"url":"https://github.com/cloudutil/AutoSpotting","last_synced_at":"2025-04-05T09:04:36.286Z","repository":{"id":37712307,"uuid":"63696142","full_name":"LeanerCloud/AutoSpotting","owner":"LeanerCloud","description":"Saves up to 90% of AWS EC2 costs by automating the use of spot instances on existing AutoScaling groups. Installs in minutes using CloudFormation or Terraform. Convenient to deploy at scale using StackSets. Uses tagging to avoid launch configuration changes. Automated spot termination handling. Reliable fallback to on-demand instances.","archived":false,"fork":false,"pushed_at":"2024-10-15T13:34:15.000Z","size":31606,"stargazers_count":2332,"open_issues_count":44,"forks_count":312,"subscribers_count":58,"default_branch":"master","last_synced_at":"2024-10-29T17:11:45.355Z","etag":null,"topics":["amazon-web-services","automation","autoscaling","autoscaling-groups","aws","aws-autoscaling","aws-lambda","cheaper-spot-instances","cloudformation","cost","ec2","ec2-instance","ec2-spot","go","golang-application","infrastructure","spot-instances","terraform-module"],"latest_commit_sha":null,"homepage":"https://autospotting.io","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"osl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/LeanerCloud.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["cristim"],"patreon":"cristim","custom":["https://www.paypal.me/cristim"]}},"created_at":"2016-07-19T13:27:31.000Z","updated_at":"2024-10-28T08:14:44.000Z","dependencies_parsed_at":"2024-11-06T13:05:50.662Z","dependency_job_id":"04c4a2b4-1565-4bbe-88ad-a60253e4d7ab","html_url":"https://github.com/LeanerCloud/AutoSpotting","commit_stats":{"total_commits":623,"total_committers":70,"mean_commits":8.9,"dds":0.6484751203852328,"last_synced_commit":"9bc88ee72f9587a2de187cbd333fc06452b4e846"},"previous_names":["autospotting/autospotting","cloudutil/autospotting"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LeanerCloud%2FAutoSpotting","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LeanerCloud%2FAutoSpotting/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LeanerCloud%2FAutoSpotting/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/LeanerCloud%2FAutoSpotting/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/LeanerCloud","download_url":"https://codeload.github.com/LeanerCloud/AutoSpotting/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247312077,"owners_count":20918344,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amazon-web-services","automation","autoscaling","autoscaling-groups","aws","aws-autoscaling","aws-lambda","cheaper-spot-instances","cloudformation","cost","ec2","ec2-instance","ec2-spot","go","golang-application","infrastructure","spot-instances","terraform-module"],"created_at":"2025-04-05T09:02:40.738Z","updated_at":"2025-04-05T09:04:36.253Z","avatar_url":"https://github.com/LeanerCloud.png","language":"Go","funding_links":["https://github.com/sponsors/cristim","https://patreon.com/cristim","https://www.paypal.me/cristim","https://www.patreon.com/cristim/overview"],"categories":["Repositories"],"sub_categories":[],"readme":"\u003c!-- markdownlint-disable MD003 MD026 MD033 --\u003e\n\n## AutoSpotting - Community Edition ##\n\nMost new features, bug fixes and improvements for increased reliability and better performance developed by the main author after July 2022 are\nonly available in the commercial version of AutoSpotting, available on the AWS \n[Marketplace](https://aws.amazon.com/marketplace/pp/prodview-6uj4pruhgmun6). The only exception are security\nfixes, which will be patched in the Community Edition on a best effort basis.\n\nIf you notice any issues in the Community Edition feel free to report them but they are not being worked on\nby LeanerCloud anymore. Chances are they are already solved in the Commercial Edition, so you may want to \ntry that out.\n\nBug fixes and new features contributed by the community are welcome and will\nremain available in the Community Edition indefinitely.\n\nThe list of ehnancements in the Commercial edition is documented in detail \n[here](https://github.com/LeanerCloud/AutoSpotting/discussions/489), but you can see below a brief feature comparison of AutoSpotting options vs. AWS AutoScaling groups.\n\n| **Feature**                                                | **Autoscaling Groups**              | **AutoSpotting Community Edition**             | **AutoSpotting Commercial**        |\n|------------------------------------------------------------|-------------------------------------|------------------------------------------------|------------------------------------|\n| Easy configuration rollout using tags                      | ❌                                  | ✅                                             | ✅                                 |\n| Automated Spot instance type diversification               | If using attribute based selection  | ✅                                             | ✅                                 |\n| Allocation policy with bias for new instance types         | ❌                                  | ✅                                             | ✅                                 |\n| Spot event handling - Rebalancing Recommendation           | Optional                            | ✅ (Default - increased instance churn!)       | Optional                           |\n| Spot event handling - Termination Notification             | ❌                                  | Optional                                       | ✅ (Default - less instance churn) |\n| Spot event handling - Replacement with a new Spot instance | Requires Rebalancing Recommendation | ✅ Via Spot termination + ASG OnDemand launch  | ✅ Proactive                       |\n| Load balancer and Target Group TCP connection draining     | ✅                                  | ✅ (Relying on Autoscaling)                    | ✅ Proactive                       |\n| Automated ECS task draining                                | Requires UserData script changes    | Requires UserData script changes               | ✅                                 |\n| Failover to on-demand instances                            | ❌                                  | ✅ Relies on Autoscaling to Launch replacement | ✅ Proactive                       |\n| Instance type diversification for On-Demand Failover       | ❌                                  | ❌ (Single instance type from Launch Template) | ✅ (Same diversification as Spot)  |\n| Automated Spot product handling                            | ❌                                  | ❌                                             | ✅                                 |\n| Main deployment region                                     | N/A                                 | ❌ (N. Virginia only)                          | ✅ (Any region)                    |\n| Parallel instance type replacement                         | N/A                                 | ❌                                             | ✅                                 |\n| Recent instance type coverage                              | N/A                                 | ❌                                             | ✅                                 |\n| Auto-update instance type information                      | If using attribute based selection  | ❌                                             | Optional                           |\n| Savings reports email                                      | ❌                                  | ❌                                             | ✅                                 |\n| Relax EBS bandwidth checks for increased diversification   | N/A                                 | ❌                                             | ✅                                 |\n| Reuse ASG Launch Template when launching Spot instances    | N/A                                 | ❌                                             | ✅                                 |\n| Support Autoscaling Groups with a Mixed Instances Policy   | ✅                                  | ❌ (silently fails for such groups)            | ✅                                 |\n| Costs                                                      | Free (AWS Service)                  | Free and open source                          | 5% of savings from OnDemand        |\n\n\n\n\u003cimg src=\"logo.png\" width=\"150\" align=\"right\"\u003e\n\n[![slack](https://img.shields.io/badge/slack-chat-brightgreen.svg?logo=slack)](https://join.slack.com/t/leanercloud/shared_invite/zt-xodcoi9j-1IcxNozXx1OW0gh_N08sjg)\n[![GoReportCard](https://goreportcard.com/badge/github.com/LeanerCloud/AutoSpotting)](https://goreportcard.com/report/github.com/LeanerCloud/AutoSpotting)\n[![Patreon](https://img.shields.io/badge/patreon-donate-yellow.svg)](https://www.patreon.com/cristim/overview)\n\n# About AutoSpotting\n\n\nAutoSpotting is the leading open source spot market automation tool, optimized\ntowards quick/easy/frictionless adoption of EC2 Spot instances at any scale.\n\nSince its launch in 2016, it's being used by thousands of users around the world,\ncompanies of all shapes and sizes, in aggregate saved them in the \nhundreds of millions of dollars so far as per our current estimations.\n\nIt is usually set up to monitor existing long-running AutoScaling groups,\nreplacing their instances with Spot instances, with minimal configuration\nchanges.\n\nOften all it needs is just tagging the AutoScaling groups with `spot-enabled=true`,\n(in some cases even that can be avoided), yielding the usual 60%-90% Spot cost\nsavings but in a better integrated and easier to adopt way\nthan other alternative tools and solutions.\n\nIt is particularly useful if you have a large footprint that you want to migrate\nto Spot quickly due to management pressure but with minimal effort and configuration\nchanges.\n\n## Demo\n\nHere's a demo of AutoSpotting and the way it integrates with the new [Spot Savings Estimator GUI](https://github.com/LeanerCloud/savings-estimator). (Click to open on YouTube)\n\n[![Demo of AutoSpotting with the Savings Estimator](https://img.youtube.com/vi/VXfCOXXtLwA/maxresdefault.jpg)](https://youtu.be/VXfCOXXtLwA \"AutoSpotting and Savings Estimator demo\")\n\n## Guiding principles ##\n\n- Customer-focused, designed to maximize user benefits and reduce adoption friction\n- Safe and secure, hosted in your AWS account and with minimal required set of IAM permissions\n- Auditable OSS code base developed in the open\n- Inexpensive, easy to install and supported builds offered through the AWS Marketplace,\nwith comprehensive support and additional features geared towards production usage at scale.\n- Simple, minimalist implementation\n\n## Benefits compared to alternatives ##\n\n- automated failover to OnDemand instances in the event of lost Spot capacity\n- automated instance type selection for Spot diversification without configuration changes from a standard OnDemand setup, and no vendor lock-in\n- bias towards cheapest AND newest instance types\n\n## How does it work? ##\n\nOnce installed and enabled to run against existing on-demand\nAutoScaling groups, AutoSpotting gradually replaces their on-demand instances\nwith cheaper spot instances that are at least as large and identically\nconfigured to the group's members, without changing the group launch\nconfiguration in any way. You can also keep running a configurable number of\non-demand instances given as percentage or absolute number and it automatically\nfails over to on-demand in case of spot instance terminations.\n\nGoing forward, as well as on any new ASGs that match the expected tags, any new\non-demand instances above the amount configured to be kept running will be immediately\nreplaced with spot clones within seconds of being launched.\n\nIf this fails temporarily due to insufficient spot capacity, AutoSpotting will\ncontinuously attempt to replace them every few minutes until successful after\nspot capacity becomes available again.\n\nWhen launching Spot instances, the compatible instance types are chosen by\ndefault using a the\n[capacity-optimized-prioritized](https://docs.amazonaws.cn/en_us/AWSEC2/latest/UserGuide/ec2-fleet-examples.html#ec2-fleet-config11)\nallocation strategy, which is given a list of instance types sorted by price. This\nconfiguration offers a good tradeoff between low cost and significantly reduced\ninterruption rates. The lowest-price allocation strategy is still available as a\nconfiguration option.\n\nA single installation can handle all enabled groups from an entire AWS account in\nparallel across all available AWS regions, but it can be restricted to fewer\nregions if desired in certain situations.\n\nYour groups will then monitor and use these Spot instances just like they would\ndo with your on-demand instances. They will automatically join their respective\nload balancer and start receiving traffic once passing the health checks, and\nthe traffic would automatically be drained on termination.\n\nSee this video for more imformation on AutoSpotting.\nhttps://youtu.be/foobAmWpexI\n\n## What savings can I expect? ##\n\nThe savings it generates are in the 60-90% range usually seen when using spot\ninstances, but they may vary depending on region and instance type.\n\n## What's under the hood? ##\n\nThe entire logic described above is implemented in a set of Lambda functions\ndeployed using CloudFormation or Terraform stacks that can be installed and\nconfigured in just a few minutes.\n\nThe stack uses the minimal set of IAM permissions required for them to\nwork and requires no admin-like cross-account permissions. The entire code base\ncan be audited to see how these permissions are being used and even locked down\nfurther if your audit discovers any issues. **This is not a SaaS**, there's no\ncomponent that calls home or reveals any details about your infrastructure.\n\nThe main Lambda function is written in the Go programming language and the code\nis compiled as a static binary. As of August 2021 this has been included in a\nDocker image used by the Lambda function.\n\nThe stack also consists of a few CloudWatch event triggers, that run the Lambda\nfunction periodically and whenever it needs to take action against the enabled\ngroups. Between runs your group is entirely managed by AutoScaling (including\nany scaling policies you may have) and load balancer health checks, that can\ntrigger instance launches or replacements using the original on-demand launch\nconfiguration.\n\nRead [here](TECHNICAL_DETAILS.md) for more information and implementation\ndetails.\n\n## FAQs ##\n\nMany Frequently Asked Questions about the project are answered in the\n[FAQ](https://autospotting.io/faq), *please read this first before\nasking for support*.\n\nIf you have additional questions not covered there, reach out to us on \n[Slack](https://join.slack.com/t/leanercloud/shared_invite/zt-xodcoi9j-1IcxNozXx1OW0gh_N08sjg)\nand we're happy to help.\n\n## Getting Started ##\n\nIt's usually as easy as launching a CloudFormation (or\n[Terraform](https://github.com/AutoSpotting/terraform-aws-autospotting)) stack\nand setting the (configurable) `spot-enabled` tag on the AutoScaling groups\nwhere you want it enabled to `true`.\n\nWhen installed from the AWS\n[marketplace](https://aws.amazon.com/marketplace/pp/prodview-6uj4pruhgmun6), all\nthe required infrastructure and configuration will be created automatically, so\nyou can get started as fast as possible. Otherwise you'll need to build it\nyourself as per the instructions available [here](CUSTOM_BUILDS.md).\n\nFor more detailed information on how to get started you can also read this\n[document](START.md)\n\n## Support ##\n\nMarketplace subscribers can get support on [Slack](https://join.slack.com/t/leanercloud/shared_invite/zt-xodcoi9j-1IcxNozXx1OW0gh_N08sjg)\nand any feature requests or issues raised via this communication channel\nwill be prioritized.\n\nCommunity support is available to OSS users on the\n[gitter](https://gitter.im/cristim/autospotting) chat room, where the main\nauthors and other users are likely to help you solve issues. This is offered on\na best effort basis and under certain conditions, such as using the latest\nversion of the software available on the main Github branch, without any code\ncustomizations and using the default configuration options.\n\nIf you need help for a large scale rollout or migrating from alternative\ntools/solutions get in touch on [gitter](https://gitter.im/cristim).\n\n## Contributing ##\n\nAutoSpotting is open source and developed in the open by a vibrant\ncommunity of dozens of contributors.\n\nThe Community Edition is open for contributions, submitted according to the contribution\n[guidelines](CONTRIBUTING.md).\n\nIndividuals and companies supporting the development of the open source code\nget free of charge support in getting their code merged upstream.\n\n### Community Edition ###\n\nThe source code from this repo is and will always be open source, so you can build and run\nit yourself, see how it works and even enhance it if you want.\n\nAny external contributions to the Community Edition will be remain available to all\nCommunity Edition users, but also included in future versions of the commercial edition.\n\n### Subscriptions ###\n\nA free low traffic mailing list is available on [Beehiiv](https://leanercloud.beehiiv.com/), where\nyou can sign up for occasional emails related to the project, mainly related to\nmajor changes in the open source code, savings tips or announcements about other\ntools I've been working on.\n\nAnnouncements on new Marketplace releases, including comprehensive release\nnotes, upgrade instructions and tips to get the most out of AutoSpotting will be\ncommunicated in private to Patreon\n[subscribers](https://www.patreon.com/cristim/overview).\n\nA Github [sponsors](https://github.com/sponsors/cristim) subscription is also\navailable for people interested in the ongoing development of AutoSpotting, with\ntiers covering anything from a non-strings attached donation, prioritization of\nfeature requests, all the way to custom features development and maintenance of\nprivate customized forks.\n\nPlease get in touch on [gitter](https://gitter.im/cristim) if you have any\nquestions about these offerings or if you have any other ideas on how I could\nprovide additional value to my community.\n\n## Compiling and Installing ##\n\nIt is recommended to use the commercial offering available on the\nAWS marketplace, which is more feature-complete, easier to install, supports\nfurther development of the software and includes comprehensive support.\n\nBut if you have some special needs that require some customizations,\nyou can always build and run your customized binaries\nthat you maintain on your own.\n\nMore details are available [here](CUSTOM_BUILDS.md)\n\n## License ##\n\nThis software is distributed under the terms of the OSL-3.0 [license](LICENSE).\n\nThe AWS Marketplace offering is made available under the standard AWS\nMarketplace EULA.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudutil%2FAutoSpotting","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudutil%2FAutoSpotting","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudutil%2FAutoSpotting/lists"}