{"id":19551095,"url":"https://github.com/ksingh7/openspot","last_synced_at":"2025-04-26T20:31:07.811Z","repository":{"id":49902583,"uuid":"379862818","full_name":"ksingh7/openspot","owner":"ksingh7","description":"Run OpenShift (CRC) on AWS Spot Instances","archived":false,"fork":false,"pushed_at":"2022-11-30T12:04:55.000Z","size":3875,"stargazers_count":21,"open_issues_count":3,"forks_count":9,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-04T17:47:51.804Z","etag":null,"topics":["aws","code-ready-containers","openshift","spot-instance"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ksingh7.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-06-24T08:53:37.000Z","updated_at":"2024-07-09T01:26:21.000Z","dependencies_parsed_at":"2023-01-23T19:01:21.853Z","dependency_job_id":null,"html_url":"https://github.com/ksingh7/openspot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksingh7%2Fopenspot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksingh7%2Fopenspot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksingh7%2Fopenspot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksingh7%2Fopenspot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ksingh7","download_url":"https://codeload.github.com/ksingh7/openspot/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251051327,"owners_count":21528787,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","code-ready-containers","openshift","spot-instance"],"created_at":"2024-11-11T04:10:09.365Z","updated_at":"2025-04-26T20:31:06.863Z","avatar_url":"https://github.com/ksingh7.png","language":"Shell","readme":"![OpenSpot](assets/openspot.gif)\n## tldr;\n1. Make sure AWS CLI is configured properly with right set of credentials and access to launch resources\n2. Execute the following to launch CRC on AWS Spot Instance\n```\ngit clone https://github.com/ksingh7/openspot.git\ncd openspot/aws\ntime bash launch.sh -r ap-south-1 -a ap-south-1a -v false\n```\n## Preamble\n### What is crc\nRed Hat CodeReady Containers (CRC) brings a minimal single node OpenShift 4 cluster to your local computer. This cluster provides a minimal OpenShift environment for development and testing purposes. [Read more](https://developers.redhat.com/products/codeready-containers/overview)\n\n### What are Spot Instances\nAmazon EC2 Spot Instances let you take advantage of unused EC2 capacity in the AWS datacenter. Spot Instances are available at up to a 90% discount compared to On-Demand prices. [Read more](https://aws.amazon.com/ec2/spot/)\n### What is OpenSpot\nOpenSpot [Open~~Shift on~~ Spot ~~Instance~~] is a tool that helps you deploy CRC on AWS Spot Instances in a fully automated \u0026 resilient manner.\nThe core idea of openspot is automate all the steps required to launch AWS Spot Instance and automatically configure CRC.\nOpenSpot allows you to tear down CRC spot instance and their associated resources, when you do not need it, that translates to further cost saving.\n\nFor example : You are a developer/architect, need a CRC environment daily for 4 hours. You can use OpenSpot, launch your CRC environment and terminate after 4 hours, and just repeat this step daily. OpenSpot uses EBS volume to persist your CRC instance, destroying and re-creating OpenSpot instance, do not destroy the CRC VM that you are working on daily (unless you are deleting EBS volume)\n\n##### Features of OpenSpot\n- Select Region \u0026 AZ of your choice\n- Sequentially provision all AWS resources required (as pre-requisite)\n  - Key-pair\n  - IAM role, policy, instance-policy\n  - Security Group\n  - user-data as Template\n- Launch Spot Instnace\n- Configure OS\n  - Install all required packages\n  - Get CRC/OC binaries\n- EBS volume\n  - Provision, Attach\n  - Detect/Create/Mount filesystem\n- CRC Setup\n    - Dynamically set CPU/Memory \n    - Expand CRC root disk\n- Configure Haproxy \u0026 make CRC instance available remotely\n- Handle Spot Instance Termination\n    - Provision new Spot Instnace \n    - Detect previous instance of CRC and resume that\n- Cleanup\n  - Destroy all AWS resources requested by openspot\n\n![OpenSpot Technical Architecture](assets/openspot-tech-arch.png)\n## Setup\n### Prerequisite\n- AWS CLI must be configured on local machines\n- AWS Admin Access \u0026 Secret Key\n  - If you do not have Admin access, make sure your AWS ID has right capabilites to  provision resources like `IAM roles, policies,instance-policy,key-pair,security group,spot-instance,EBS`\n- Test configuration of AWS CLI\n```\naws ec2 describe-instances\n```\n- Get OpenSpot\n```\ngit clone https://github.com/ksingh7/openspot.git\ncd openspot/aws\n```\n## Deploy CRC on Spot Instance\n\n```\n    usage launch.sh -r \"AWS_Region_Name\" -a \"AWS_AZ_NAME\" -v \"true or false\"\n    OPTIONS\n    -r \"AWS Region Name : Optional, if not provided, will use AWS CLI default value\"\n    -a \"AWS Availablity Zone Name : Optional, if not provided, will use AWS CLI default value\"\n    -v \"Optional : Verbose Output, set either true or false, default value is false\"\n    -h \"Show help menu\"\n    -i \"AMI ID\"\n```\n- To launch AWS Spot Instance in `ap-south-1` region and `ap-south-1a` availablity zone, execute\n```\ntime bash launch.sh -r ap-south-1 -a ap-south-1a -v false\n```\nSample output\n```\n$ time bash launch.sh -r ap-south-1 -a ap-south-1a -v false\nNeed your SSH Public Key absolute path to create AWS Key Pair in the selected Region (ex: /Users/karasing/.ssh/id_rsa.pub) :\nEnter SSH Public Key Path [/Users/karasing/.ssh/id_rsa.pub]:\nNew key-pair named crc-key-pair created in region ap-south-1...\nCreating IAM Role ...\nAdding policy to IAM Role ...\nCreating Instance Profile ...\nAdding Role to Instance Profile ...\nCreating Security Group ...\nGenerating User-Data script file ...\nGenerating Launch Specification file ...\nLaunching SPOT Instance, Please Wait ...\nPlease allow 5 minutes for instance configuration\nTrying to tail instance setup logs ...\nApplying TAG to Instance\nWarning: Permanently added '13.232.50.164' (ECDSA) to the list of known hosts.\nInstalling required packages ... [Done]\nSetting up AWS Cli... [Done]\nUsing existing EBS Volume ...\n...\n...\nStarted the OpenShift cluster.\n\nThe server is accessible via web console at:\n  https://console-openshift-console.apps-crc.testing\n\nLog in as administrator:\n  Username: kubeadmin\n  Password: kubeadmin\n\nLog in as user:\n  Username: developer\n  Password: developer\n\nUse the 'oc' command line interface:\n  $ eval $(crc oc-env)\n  $ oc login -u developer https://api.crc.testing:6443\nSetting up HAPROXY on host machine ...\nStarting HAPROXY Service ...\n========= Post Launch Configuration Completed Successfully ==============\n===== You can now Exit from logs tail command by presing Ctrl+C ====\nbash launch.sh -r ap-south-1 -a ap-south-1a -v false  \n9.87s user 12.99s system 3% cpu 9:55.69 total\n```\n## Configure Local Machine to remotely connect to CRC Instance\n- Automated Instructions for Macos Client to connect to CRC Instance\n```\nbash aws/mac_client.sh\n```\n- Manual Instructions for configuring `MacOS` client to connet to remote CRC\n```\nbrew install dnsmasq\nmkdir -p /usr/local/etc/dnsmasq.d\ntouch /usr/local/etc/dnsmasq.d/crc.conf\nEIP=$(aws ec2 describe-instances --filters \"Name=instance-type,Values=c5n.metal\"  --query \"Reservations[*].Instances[*].PublicIpAddress\" --output=text) ; \necho \"address=/apps-crc.testing/$EIP\" \u003e /usr/local/etc/dnsmasq.d/crc.conf ;\necho \"address=/api.crc.testing/$EIP\" \u003e\u003e /usr/local/etc/dnsmasq.d/crc.conf ;\nsudo brew services restart dnsmasq ;\ndig apps-crc.testing @127.0.0.1 ;\ndig console-openshift-console.apps-crc.testing @127.0.0.1 ;\n```\n- Login to OC\n```\noc login -u kubeadmin -p kubeadmin https://api.crc.testing:6443\noc login -u developer -p developer https://api.crc.testing:6443\n```\n- Manual Instructions for Linux `Fedora`\n```\nsudo dnf install dnsmasq\n\nsudo tee /etc/NetworkManager/conf.d/use-dnsmasq.conf \u0026\u003e/dev/null \u003c\u003cEOF\n[main]\ndns=dnsmasq\nEOF\n\nEIP=$(aws ec2 describe-instances --filters \"Name=instance-type,Values=c5n.metal\"  --query \"Reservations[*].Instances[*].PublicIpAddress\" --output=text) ; \n\nsudo tee /etc/NetworkManager/dnsmasq.d/crc.conf \u0026\u003e/dev/null \u003c\u003cEOF\naddress=/apps-crc.testing/$EIP\naddress=/api.crc.testing/$EIP\nEOF\n\nsudo systemctl reload NetworkManager\n```\n- Connect to CRC OpenShift running on Spot Instance using OpenShift CLI `oc`\n```\nDeveloer Account\n-----------------\noc login -u developer -p developer https://api.crc.testing:6443\n\n# Open OpenShift Console in your local browser\n# URL : https://console-openshift-console.apps-crc.testing\n# username : developer\n# password : developer\n\nKubeadmin Account\n------------------\nssh fedora@$EIP crc console --credentials\n# Get oc login command for kubeadmin user\n```\n  - OpenShift console URL `https://console-openshift-console.apps-crc.testing`\n## Destroy your CRC Spot Instance\nWhen you are not using your CRC instance provisioned using OpenSpot, you can destroy that to save cost.\n\nYou can choose to destroy just the AWS Spot Instance and keep the EBS volume if you want to restart your CRC instance and resume where you left last time Or you can destroy both Spot Instance and EBS volume.\n```\n    usage destroy.sh -r \"AWS_Region_Name\" -a \"AWS_AZ_NAME\" -d \"true or false\" -v \"true or false\"\n    OPTIONS\n    -r \"AWS Region Name : Optional, if not provided, will use AWS CLI default value\"\n    -a \"AWS Availablity Zone Name : Optional, if not provided, will use AWS CLI default value\"\n    -d \"Delete EBS Volume (true or false), default = false\"\n    -v \"Optional, used for verbose output (true or false), default = false\"\n    -h \"Show help menu\"\n```\n- To destroy just the AWS Spot Instance\n```\nbash destroy.sh -r ap-south-1 -a ap-south-1a -d false -v false\n```\n- To destroy both AWS Spot Instance and EBS volume\n```\nbash destroy.sh -r ap-south-1 -a ap-south-1a -d true -v false\n```\n- Sample Output\n```\n$ time bash destroy.sh -r ap-south-1 -a ap-south-1a -d false -v false\nDeleting CRC resources in Region : ap-south-1 ...\nDeleting CRC resources in AZ : ap-south-1a ...\nTerminating up Spot Instance ...\nDeleting Spot Request ...\nDetaching volume\nRemoving Role from Instance Profile ... [Done]\nDeleting Role Policy ... [Done]\nDeleting Role ... [Done]\nDeleting Instance Profile ... [Done]\nDeleting Key Pair ... [Done]\nDeleting Security Group... [Done]\nbash destroy.sh -r ap-south-1 -a ap-south-1a -d false -v false  \n7.08s user 8.30s system 62% cpu 24.693 total\n```\n## Appendix\n- Time calculation\n   - Time it takes to launch everything from scratch : ~15 minutes\n   - Time it takes to destroy everything  : ~7 minutes\n   - Time it takes to launch CRC on existing EBS volume : ~10 minutes\n\n- Cost Calculation\n  - AWS Spot Instance cost for c5n.metal instance type : $0.90 / Running Hour\n  - AWS EBS cost for 100GB of gp2 : $11.40\n\n\n- SSH into the instance\n```\nEIP=$(aws ec2 describe-instances --filters \"Name=instance-type,Values=c5n.metal\"  --query \"Reservations[*].Instances[*].PublicIpAddress\" --output=text) ; \nssh fedora@$EIP\n```\n- Check progress of Instance setup \n```\nssh  fedora@$EIP cat /var/log/crc_status\nssh  fedora@$EIP tail -f /var/log/crc_setup.log\nssh  fedora@$EIP tail -f /var/log/cloud-init-output.log\n```\n### Todo\n- Add support for GCP preemptible instances\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksingh7%2Fopenspot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fksingh7%2Fopenspot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksingh7%2Fopenspot/lists"}