{"id":13821208,"url":"https://github.com/yahoo/ariel","last_synced_at":"2025-05-05T06:32:15.069Z","repository":{"id":43288082,"uuid":"191477841","full_name":"yahoo/ariel","owner":"yahoo","description":"Ariel is an AWS Lambda designed to collect, analyze, and make recommendations about Reserved Instances for EC2.","archived":false,"fork":false,"pushed_at":"2022-12-08T22:47:41.000Z","size":69,"stargazers_count":67,"open_issues_count":0,"forks_count":13,"subscribers_count":12,"default_branch":"master","last_synced_at":"2025-04-08T18:51:37.714Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/yahoo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"Code-of-Conduct.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-06-12T01:50:59.000Z","updated_at":"2025-01-10T06:34:36.000Z","dependencies_parsed_at":"2023-01-25T15:15:40.228Z","dependency_job_id":null,"html_url":"https://github.com/yahoo/ariel","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Fariel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Fariel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Fariel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/yahoo%2Fariel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/yahoo","download_url":"https://codeload.github.com/yahoo/ariel/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252451775,"owners_count":21749980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-04T08:01:17.597Z","updated_at":"2025-05-05T06:32:14.741Z","avatar_url":"https://github.com/yahoo.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Ariel\n\n\u003e Ariel is an AWS Lambda designed to collect, analyze, and make\nrecommendations about Reserved Instances for EC2.\n\n## Table of Contents\n- [Background](#backgorund)\n- [Installation](#installation)\n- [Configuration](#configuration)\n  - [Dependencies](#dependencies)\n  - [Optional Functionality](#optional-functionality)\n  - [Recommendations Algorithm](#recommendations-algorithm)\n  - [Outputs](#outputs)\n- [Usage](#usage)\n  - [Additional Considerations](#additional-considerations)\n- [Contributing](#contributing)\n- [Maintainers](#maintainers)\n- [License](#license)\n\n## Background\n\nAriel is a tool designed to make purchase recommendations for Reserved\nInstances for EC2. There are many tools that currently provide similar functionality to generate recommendations for either all accounts, where there is no indication of which account will benefit from the Reserved Instance, or for each account individually, which does not properly take into consideration Reserved Instance float, resulting in a recommendation of too many Reserved\nInstances.\n\nThe main benefit of Ariel, and what all other tools appear to lack, is the ability to combine these two approaches. Reserved Instances demand is calculated based on overall company usage, but specific recommendations are made by account such that an individual account will never purchase more than it is currently using, and also the combine Reserved Instances total will never exceed the current demand for the whole company.\n\nAriel also has more configuration options to allow finer tuning of the\npurchase recommendation algorithm; additional details can be found\nbelow in the Algorithms section.\n\n## Installation\n\nPlease refer to [INSTALL.md](INSTALL.md) for installation instructions.\n\n## Configuration\n\nConfiguration inputs are documented in [config-example.yaml](config-example.yaml).\nIt is expected that you will:\n- Copy and modify this file for your specific environment\n- Deploy this configuration file to S3\n- Configure the Lambda input parameter to point to this file in S3\n\n### Dependencies\n\nThere are many components and dependencies for Ariel that allow different deployments.\n\n**Master Billing Account - Reserved Instances**\n\nAriel needs access to the Master Billing Account to collect all active\nReserved Instances.  This is collected through the Reserved Instances\nUtilization report provided by Cost Explorer.  In addition to this\nreport including Reserved Instances information about all active\naccounts in your Organization, this is one of the only ways to collect\ninformation about Reserved Instances in suspended accounts in your\nOrganization.  While suspended accounts are otherwise inactive, their\nReserved Instances are still active with their monthly fee being charged\nto the otherwise closed account, and they are still able to float to any\naccount linked to your Master Billing Account.\n\n**Master Billing Account - Account Names**\n\nAriel can be configured to gather account names from the Organizations\nAPI only accessible from the Master Billing Account.  This access is not\nrequired, as a flat file may be used instead, or if neither are\nspecified, Accound IDs will be used instead of names.\n\n**Master Billing Account - Cost and Usage Reports + Athena**\n\nThis version of Ariel requires [Athena Optimized Cost and Usage Reports](https://aws.amazon.com/about-aws/whats-new/2018/11/aws-cost-and-usage-report-integrates-with-amazon-athena/).\nCost and Usage Reports can only be written to a bucket in the Master\nBilling Account.  Athena Optimization only works when the data is stored\nin the bucket it was originally written to.  Report data is written from\na 3rd party AWS account and can not be shared with other accounts.  The\nend result of these three statements is that Athena must be used from the\nMaster Billing Account.\n\nAccess requirements for the Master Billing Account have been separated\ninto a standalone CloudFormation template so that Ariel can run in an\naccount other than the Master Billing Account, and just use a\ncross-account role when necessary.\n\n**VPC**\n\nAriel does not need a VPC to run. However, if you want to deploy it to\na VPC you will want to create your own VPC with the necessary Subnets, Route Tables, Internet Gateway, NAT Gateways, Network ACLs, and Security Groups, we do not provide a VPC template.\n\n### Optional Functionality\n\n* **VPC** - Ariel can be deployed with or without a VPC.  Unfortunately the\nAWS::Serverless transform does not yet support conditionals, so the\nAriel VPC and non-VPC templates are separate files.\n\n* **AuroraDB/RDS** - Ariel can be configured to publish the results to a Postgres Database.\nIf using this database, it is recommended that you deploy Ariel in a VPC\nand prevent internet access directly to your database.\n\n### Recommendations Algorithm\n\nProper Reserved Instances management is an art form that is very\ndifficult and complex, and mistakes can be costly.  When making\npurchases, you really must be certain that you will use the Reserved\nInstances for at least as long as the break-even term for each\nReserved Instance. The following content describes how to fine tune your Ariel recommendations.\n\n#### Utilization vs Coverage\n\nReserved Instance utilization and coverage are inversely related\nmetrics.  As you increase your coverage, your utilization will usually\ndecrease.  Properly managing Reserved Instances involves maximizing\ncoverage, while keeping utilization high enough to be cost beneficial.\nAdditionally, since the most cost effective Reserved Instance has a\nthree year term, you will also need to predict your future usage to make\neffective purchases.\n\nFor Reserved Instances purchase recommendations in Ariel, a target\nutilization value is specified.  To achieve higher coverage, specify\na lower desired utilization.  Ariel also has a special `BREAK_EVEN`\nkeyword for utilization that will calculate the specific break even\nutilization percents for the currently evaluated region and family.\n\n#### Break Even\n\nActual break even numbers vary by region and instance type family, but\nfor a rough rule to follow for Standard Reserved Instances in US regions:\n\n* A one year Reserved Instance breaks even in about 7 months.\n* A three year Reserved Instance breaks even in about 13 months.\n* Renewing a one year Reserved Instance once will cost more than if\n  a three year Reserve Instance was purchased initially.\n\n#### Aggressive / Conservative\n\nAriel supports two thresholds that control 3 configurations for\npurchasing aggressiveness.  For both thresholds, the instances run rate,\nfor an individual instance type family, region, tenancy, and operating\nsystem, is fit to a straight line, and the threshold is applied to the\nslope of that line.\n\n* If the slope is greater than `AGGRESSIVE_THRESHOLD`, the `AGGRESSIVE`\n  configuration is used.\n* If the slope is smaller than `CONSERVATIVE_THRESHOLD`, the\n  `CONSERVATIVE` configuration is used.\n* Otherwise, the `DEFAULT` configuration is used.\n\nThese thresholds are specified in \"normalized units\" per day.  For\nexample, a value of 1000 means an average increase of 125xlarge\ninstances per day.  Normalized units can be found in the\n[Reserved Instances Documentation](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ri-modifying.html#ri-modification-instancemove)\n\n#### Standard / Convertible\n\nConvertible Reserved Instances allow for greater flexibility, but also\ncost on average 25% more than standard Reserved Instances.  Ariel tries\nto give you the ability to purchase a baseline of standard Reserved\nInstances, while simultaneously pushing up your coverage with\nconvertible Reserved Instances, reducing your risk if some workloads\nmigrate away.\n\nConvertible Reserved Instances recommendations will always be calculated\nafter standard, and will assume that the standard Reserved Instances are\nalso being purchased.\n\n##### Slush Account\n\nThere are some cases where the total Reserved Instances recommendation\nby account will be less than the overall company demand.  As a trivial\nexample, if two accounts would benefit from a single Reserved Instance\nduring different halves of the day, neither account would have sufficient\ndemand to make the purchase independently, but the company would still\nbenefit from purchasing one.\n\nIn this case, Ariel can be configured to make a purchase recommendation\nin a slush account.\n\nThe algorithm evaluates in this order:\n\n* Standard in-account\n* Standard slush account\n* Convertible in-account\n* Convertible slush account\n\n#### Regional / Availability Zone\n\nWith the introduction of [OnDemand Capacity Reservations](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-capacity-reservations.html),\nthere is no longer any benefit from ever using Availability Zone based\nReserved Instances.  This support has been removed from Ariel, and\nRegional Reserved Instances will always be recommended.\n\n### Outputs\n\nPlease refer to [REPORTS.md](REPORTS.md) for detailed information about\nthe reports generated.\n\n## Usage\n\nAriel is typically run daily.  It's possible to configure it to run\nmore frequently, but this provides little additional value, as Cost\nand Usage Reports are only updated two to three times per day.  There\nare no concerns with running Ariel less frequently.\n\nSome of the report data is useful to look at every day, such as the\nri-usage report to find available excess Reserved Instances for shifting\nworkloads that are easy to move across instance types.  The main report\nfrom Ariel, ri-purchases, tends to have a slower cadence.  It is\nrecommended to evaluate a single ri-purchases report monthly or\nquarterly to make decisions about which recommendations to purchase.\nThis review and decision making process is necessarily manual.\n\nAssuming that a purchase is completed through a support case or your\nConcierge, it will take some time for your order to be processed.  Ariel\nwill automatically pick up new Reserved Instances once they have been\npurchased through the report accessed from Cost Explorer.\n\n### Opening a support case\n\nIf your Support Plan allows for it, you can bulk purchase Reserved Instances\nthough a support case.\n\n* Case Type: `Account and billing support`\n* Type: `Billing`\n* Category: `Reserved Instances`\n* Severity: `General question`\n\nYou will want to download the [AWS RI Transaction Request Worksheet](https://s3.amazonaws.com/awsreservedinstancetransactionrequestworksheet/AWS+RI+Transaction+Request+Worksheet.xlsb), fill it out, and attach it to the case.\n\nFor the account list, you may want to `cat ri-purchases.csv | cut -f1 -d, | grep -v Account | sort -u \u003e accounts.txt`\n\nFor the purchase sheet, you can load ri-purchases.csv into Excel, then copy the columns A-J, excluding the header, then `Paste Values` into the worksheet.  You will need to manually populate the `Desired Start Date` field.\n\n### Additional Considerations\n\nAriel does not yet make any recommendations related to modifying\nexisting convertible Reserved Instances.  Before making a purchase\nyou should:\n\n* Check the ri-usage report to find instance type families with excess\n  unused Reserved Instances\n* Check the reserved instances report to find matching convertible\n  Reserved Instances\n* Modify the unused Reserved Instances to cover new recommended Reserved\n  Instances\n* Subtract the modified Reserved Instances from the recommendation\n\n## Contributing\n\nPlease refer to [CONTRIBUTING.md](CONTRIBUTING.md) for information about\nhow to get involved. We welcome issues, questions, and pull requests.\nPull Requests are welcome.\n\n## Maintainers\nSean Bastille: sean.bastille@verizonmedia.com\nMicah Meckstroth: micah.meckstroth@verizonmedia.com\n\n## License\nThis project is licensed under the terms of the\n[Apache License, Version 2.0](LICENSE) open source license. Please refer\nto [LICENSE](LICENSE) for the full terms.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyahoo%2Fariel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyahoo%2Fariel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyahoo%2Fariel/lists"}