{"id":19168387,"url":"https://github.com/bloomberg/hackathon-aws-cluster","last_synced_at":"2025-06-30T02:04:45.398Z","repository":{"id":66125181,"uuid":"75961915","full_name":"bloomberg/hackathon-aws-cluster","owner":"bloomberg","description":null,"archived":false,"fork":false,"pushed_at":"2018-01-31T19:55:17.000Z","size":2716,"stargazers_count":11,"open_issues_count":0,"forks_count":15,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-03T23:15:34.994Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bloomberg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-12-08T17:56:36.000Z","updated_at":"2023-11-09T01:12:03.000Z","dependencies_parsed_at":"2023-04-01T14:19:13.857Z","dependency_job_id":null,"html_url":"https://github.com/bloomberg/hackathon-aws-cluster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Fhackathon-aws-cluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Fhackathon-aws-cluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Fhackathon-aws-cluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bloomberg%2Fhackathon-aws-cluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bloomberg","download_url":"https://codeload.github.com/bloomberg/hackathon-aws-cluster/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240249290,"owners_count":19771641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-09T09:42:29.616Z","updated_at":"2025-02-22T23:27:13.706Z","avatar_url":"https://github.com/bloomberg.png","language":"HTML","funding_links":[],"categories":[],"sub_categories":[],"readme":"# git-hackathon-vpc\n\nThis repository contains Python scripts and CloudFormation templates used to launch a cluster of user instances for a Git hackathon. The Python scripts\nrequire Python 3 and the 'boto3' module (install with 'pip' or any other mechanism you like).\n\n## Setup\n\nYou will need an AWS access key (both \"access key id\" and \"secret access key\") with sufficient privileges in the region you wish to use in order to create a variety of AWS objects. These credentials\nshould be set as environment variables ```AWS_ACCESS_KEY_ID``` and ```AWS_SECRET_ACCESS_KEY```.\n\n## git-vpc.json\n\nThis is the 'outer' CloudFormation template; it creates the VPC, an access node on a preselected ElasticIP, various other resources, and configures the access node to provide HTTP reverse-proxy access into the user machines.\n\n## git-vpc-user.json\n\nThis is the 'inner' CloudFormation template; it creates an instance in an isolated VPC subnet and sets up everything necessary on the instance.\nThis includes generating a random password for the 'ubuntu' user (which is copied into a private S3 bucket created by the 'outer' template) and\nShellInABox setup so that the proxy on the 'outer' node can provide access to the instance.\n\n## launch-vpc.py\n\nScript which launches the 'outer' CloudFormation stack. There are some configuration parameters at the top of the script.\n\n## launch-vpc-user.py\n\nScript which launches one (or more) user CloudFormation stacks. This accepts one or two command line parameters, indicating the user numbers of the\nstacks to be launched. There are some configuration parameters at the top of the script, and the script also obtains information left in 'outputs' in the\n'outer' stack.\n\n## get-passwords.py\n\nRetrieves the randomly-generated 'ubuntu' passwords for all user instances in the stack, putting them into a 'passwords' subdirectory.\n\n## cleanup.py\n\nDestroys all the resources currently running, starting with the user stacks and ending with the 'outer' stack.\n\n## index.html and proxies.conf\n\nFiles installed into the access node to setup reverse-proxies in the Apache HTTPD running there.\n\n## Access\n\nOnce the stacks are launched, users can connect to the IP address at the top of the ```launch-vpc.py``` script with any browser. They will be presented a list of links\nto the user instances; clicking a link will open a new tab (or window) in their browser, and they will be presented with a login prompt. They can login as ```ubuntu```\nusing the randomly-generated password obtained by running the ```get-passwords.py``` script; if this has been emailed to them, they can copy-and-paste it but *MUST*\nuse the Edit/Paste menu in their browser (or similar menu item if their browser calls it something different) as the ShellInABox window will intercept all normal\n'paste' keystrokes and send them to the VM, so they will not be handled locally in the browser.\n\nOnce logged in the users will find a clone of https://github.com/git/git in the `git` directory. All dependencies to build it, and its tests and documentation, have\nbeen installed. In addition many common text editors (Emacs, Vim, Nano) are installed and various other tools. Users have ```sudo``` access and can install any\nadditional packages they find necessary or useful. ```USE_LIBPCRE``` has been set in ```.bash_profile``` so that Git will be built with PCRE support.\n\n## Typical workflow\n\n1. Clone this repository/branch.\n2. Set AWS access key environment variables.\n3. Run ```launch-vpc.py``` and wait for it to finish.\n4. Run ```launch-vpc-user.py \u003cx\u003e \u003cy\u003e``` where 'x' and 'y' define the starting and ending 'user numbers' to be launched. If 'y' is not supplied, only a single user number will be launched.\nWait for it finish.\n5. Run ```get-passwords.py```. Copy the ```passwords.txt``` file from the ```passwords``` directory somewhere that you can use it to email the passwords to the users.\n6. Run ```cleanup.py``` to destroy everything when the event is complete.\n\nIf any user instance needs to be destroyed and recreated, you will need to log in to the AWS console, navigate to the CloudFormation service, and then delete that user's stack before\nattempting to recreate it. You will need to be sure to select the proper AWS region (using the drop-down selector in the top-right corner) in the console in order to\nsee the resources created by CloudFormation.\n\n## Configuration\n\nThe user instances are quad-core VMs with 8GB of RAM, so parallel builds should work quite well.\n\n## AMI Notes\n\nThe image for the access node needs the CloudFormation tools and Apache HTTPD with mod_proxy and mod_proxy_http enabled. The cloud-init script in the CloudFormation template\nexpects the Debian/Ubuntu layout of configuration files for the web server.\n\nThe image for the user nodes needs the CloudFormation tools, the AWS CLI tool, and ShellInABox. SIAB must be configured to listen on port 80 with SSL disabled (and the SSL menu item disabled).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomberg%2Fhackathon-aws-cluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbloomberg%2Fhackathon-aws-cluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbloomberg%2Fhackathon-aws-cluster/lists"}