{"id":24911537,"url":"https://github.com/hackoregon/hackoregon-aws-infrastructure","last_synced_at":"2025-10-16T22:30:39.969Z","repository":{"id":46458717,"uuid":"84032285","full_name":"hackoregon/hackoregon-aws-infrastructure","owner":"hackoregon","description":"Hackoregon AWS Infrastructure for 2017+","archived":false,"fork":false,"pushed_at":"2021-10-11T03:56:56.000Z","size":682,"stargazers_count":6,"open_issues_count":6,"forks_count":12,"subscribers_count":21,"default_branch":"master","last_synced_at":"2025-04-09T10:52:16.396Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hackoregon.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-06T05:10:59.000Z","updated_at":"2021-11-03T16:33:32.000Z","dependencies_parsed_at":"2022-08-30T04:20:13.162Z","dependency_job_id":null,"html_url":"https://github.com/hackoregon/hackoregon-aws-infrastructure","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/hackoregon/hackoregon-aws-infrastructure","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackoregon%2Fhackoregon-aws-infrastructure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackoregon%2Fhackoregon-aws-infrastructure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackoregon%2Fhackoregon-aws-infrastructure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackoregon%2Fhackoregon-aws-infrastructure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hackoregon","download_url":"https://codeload.github.com/hackoregon/hackoregon-aws-infrastructure/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hackoregon%2Fhackoregon-aws-infrastructure/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279249044,"owners_count":26133926,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-16T02:00:06.019Z","response_time":53,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-02-02T04:20:45.313Z","updated_at":"2025-10-16T22:30:39.416Z","avatar_url":"https://github.com/hackoregon.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HackOregon 2017-2019 - Infrastructure\n\nA Set of YAML templates for deploying the HackOregon infrastructure on [Amazon EC2 Container Service (Amazon ECS)](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/Welcome.html) with [AWS CloudFormation](https://aws.amazon.com/cloudformation/). Based on the AWSLabs [EC2 Container Service Reference Architecture](https://github.com/awslabs/ecs-refarch-cloudformation) and AWS' Paul Lewis' [Fargate Reference Architecture](https://github.com/pjlewisuk/fargate-refarch-cloudformation).\n\n## Related Repositories\n\n* [Example Django Docker with CI/CD](https://github.com/hackoregon/backend-service-pattern)\n* [Example Nginix Docker Endpoint Catalog Service with CI/CD](https://github.com/hackoregon/endpoint-service-catalog)\n\n## Overview\n\n![infrastructure-overview](images/architecture-overview.png)\n\nThe repository consists of a set of nested templates that deploy the following:\n\n* A tiered [VPC](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Introduction.html) with public and private subnets, spanning an AWS region.\n* A highly available ECS cluster deployed across two [Availability Zones](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-regions-availability-zones.html) in an [Auto Scaling](https://aws.amazon.com/autoscaling/) group.\n* A pair of [NAT gateways](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/vpc-nat-gateway.html) (one in each zone) to handle outbound traffic.\n* A variety of microservice and web front-end containers deployed as [ECS services](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs_services.html).\n* An [Application Load Balancer (ALB)](https://aws.amazon.com/elasticloadbalancing/applicationloadbalancer/) to the public subnets to handle inbound traffic to the load-balanced container duplicates.\n* ALB path-based routes for each ECS service to route the inbound traffic to the correct service.\n* Centralized container logging with [Amazon CloudWatch Logs](http://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/WhatIsCloudWatchLogs.html).\n\n### Infrastructure-as-Code\n\nThis set of templates can be used to create near-identical copies of the same stack (or to use as a foundation to start a new stack).\n\nMaster templates correspond to the following deployed clusters in Hack Oregon:\n\n1. `master.yaml` - the historical \"hacko-integration\" cluster that has been used as test/staging/production since Hack Oregon's 2017 project season.\n2. (Coming soon) `master-staging.yaml` - a dedicated staging environment for all 2017+ Hack Oregon projects.  Looser access to developers, deploys from `develop` branch or equivalent in each project, limited resources to keep costs down.\n3. (Coming soon) `master-production.yaml` - a dedicated production environment for all 2017+ Hack Oregon projects.  Restricted access to developers, only deploys from `master` branch in each project, production-grade resource allocation (greater number of load-balanced tasks, higher Cpu and Memory resource allocation).\n\n### Updating and Rollback\n\nThis CloudFormation stack not only handles the initial deployment of the HackOregon infrastructure and environments, but it can also manage the whole lifecycle, including future updates. During updates, you have fine-grained control and visibility over how changes are applied, using functionality such as [change sets](https://aws.amazon.com/blogs/aws/new-change-sets-for-aws-cloudformation/), [rolling update policies](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-attribute-updatepolicy.html) and [stack policies](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/protect-stack-resources.html).\n\n## Template details\n\nThe templates below are included in this repository and reference architecture:\n\n| Template | Description |\n| --- | --- |\n| [master.yaml](master.yaml) | This is the master template - deploy it to CloudFormation and it includes all of the others automatically. |\n| [infrastructure/vpc.yaml](infrastructure/vpc.yaml) | This template deploys a VPC with a pair of public and private subnets spread across two Availability Zones. It deploys an [Internet gateway](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_Internet_Gateway.html), with a default route on the public subnets. It deploys a pair of NAT gateways (one in each zone), and default routes for them in the private subnets. |\n| [infrastructure/security-groups.yaml](infrastructure/security-groups.yaml) | This template contains the [security groups](http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPC_SecurityGroups.html) required by the entire stack. They are created in a separate nested template, so that they can be referenced by all of the other nested templates. |\n| [infrastructure/load-balancers.yaml](infrastructure/load-balancers.yaml) | This template deploys an ALB to the public subnets, which exposes the various ECS services. It is created in in a separate nested template, so that it can be referenced by all of the other nested templates and so that the various ECS services can register with it. |\n| [infrastructure/ecs-cluster.yaml](infrastructure/ecs-cluster.yaml) | This template deploys an ECS cluster to the private subnets using an Auto Scaling group. |\n| [infrastructure/rds.yaml](infrastructure/rds.yaml) | This is an example of how to deploy RDS postgres service on AWS.  We can do a Single or Multiple AZ deploy.|\n| [infrastructure/ec2-instance.yaml](infrastructure/ec2-instance.yaml) | Example of how to deploy the ec2 instances into the private subnet. The [master.yaml](masteryaml) template has examples for a bastion host and postgres db servers based on hackoregon db AMIs.|\n| [services/homelesss-service/service.yaml](infrastructure/homeless-service/service.yaml) | This is an example of the long-running Django DRF ECS service that serves a JSON API for the homelessness project. For the full source for the service, see [HackOregon Back End Service Pattern](https://github.com/hackoregon/backend-service-pattern).|\n| [services/endpoint-service/service.yaml](https://github.com/hackoregon/endpoint-service-catalog) | This is an example of a long-running Nginx ECS service that provides a static catalog of available services via the load-balanced URL.  For the full source for this service, see [HackOregon Endpoint Service Catalog](https://github.com/hackoregon/endpoint-service-catalog). |\n\nAfter the CloudFormation templates have been deployed, the [stack outputs](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/outputs-section-structure.html) contain a link to the load-balanced URLs for each of the deployed microservices.\n\n![stack-outputs](images/stack-outputs.png)\n\n## How do I...?\n\n### How to Deploy\n\nStack is setup to launch stack in the us-west-2 (Oregon) region in your account:\n\n* from the root of your copy of the repo, run `aws s3 sync . s3://hacko-infrastructure-cfn --exclude \".git/*\"`\n* copy the URL for the `master.yaml` file from S3\n* go to AWS CloudFormation - if creating new stack (e.g. for testing), choose \"create stack\"; if updating an existing stack, select that stack then click the *Update* button\n\n#### Security requirements\n\nThe account of the AWS user who initially creates the stack requires many privileges in AWS, including:\n\n* IAM Role creation\n* IAM Policy creation\n\nSubsequent incremental Updates to an existing stack can sometimes be performed by AWS users with less privileges, depending on which stack objects are being created, updated or deleted.\n\nNote: if the user attempting to perform an update doesn't have adequate permissions, CloudFormation will automatically rollback a stack change. In practice this means that if you have a change to try, try it - worst case it won't work and the stack will be left as you found it - you can't generally derail the state of the world if you lack adequate permissions.\n\n### Customize the templates\n\n1. [Fork](https://github.com/hackoregon/hackoregon-aws-infrastructure#fork-destination-box) this GitHub repository.\n2. Clone the forked GitHub repository to your local machine.\n3. Modify the templates.\n4. Upload them to an Amazon S3 bucket of your choice.\n5. Either create a new CloudFormation stack by deploying the master.yaml template, or update your existing stack with your version of the templates.\n\n### Create a new service\n\n1. Push your container to a registry somewhere (e.g., [Docker Hub](https://hub.docker.com/), [Amazon ECR](https://aws.amazon.com/ecr/)).\n2. Copy one of the existing service templates in [services/*](/services) or [fargate-services](/fargate-services).\n3. Update the `ContainerName` and `Image` parameters to point to your container image instead of the example container.\n4. Increment the `ListenerRule` priority number (no two services can have the same priority number - this is used to order the ALB path based routing rules).\n5. Duplicate one of the existing service definitions in [master.yaml](master.yaml) and point it at your new service template. Specify the HTTP `Path` at which you want the service exposed.\n6. Deploy the templates as a new stack, or as an update to an existing stack:\n    * First you'll need to create an ECR repository where the container image will (eventually) be published - these are currently just published by hand\n    * Next you'll need to create the new ECS Service - but you probably won't have a container image in ECR yet, so you won't be able to deploy an actual container just the Service and Task - so you need to set `DesiredCount` for this new service temporarily to **0**.\n    * Next you can use the deployment pipeline that uses the `ecs-deploy.sh` script [https://github.com/hackoregon/deploy-scripts/blob/master/bin/ecs-deploy.sh](here) to upload a container image to the new ECR repo\n    * Finally you can change the `DesiredCount` on the new service back to your target non-zero value and update the stack\n\n### Setup centralized container logging\n\nBy default, the containers in your ECS tasks/services are already configured to send log information to CloudWatch Logs and retain them for 365 days. Within each service's template (in [services/*](services/)), a LogGroup is created that is named after the CloudFormation stack. All container logs are sent to that CloudWatch Logs log group.\n\nYou can view the logs by looking in your [CloudWatch Logs console](https://console.aws.amazon.com/cloudwatch/home?#logs:) (make sure you are in the correct AWS region).\n\nECS also supports other logging drivers, including `syslog`, `journald`, `splunk`, `gelf`, `json-file`, and `fluentd`. To configure those instead, adjust the service template to use the alternative `LogDriver`. You can also adjust the log retention period from the default 365 days by tweaking the `RetentionInDays` parameter.\n\nFor more information, see the [LogConfiguration](http://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_LogConfiguration.html) API operation.\n\n### Change the ECS host instance type\n\nThis is specified in the [master.yaml](master.yaml) template.\n\nBy default, [t2.large](https://aws.amazon.com/ec2/instance-types/) instances are used, but you can change this by modifying the following section:\n\n```yaml\nECS:\n  Type: AWS::CloudFormation::Stack\n    Properties:\n      TemplateURL: ...\n      Parameters:\n        ...\n        InstanceType: t2.large\n        InstanceCount: 4\n        ...\n```\n\n### Adjust the Auto Scaling parameters for ECS hosts and services\n\nThe Auto Scaling group scaling policy provided by default launches and maintains a cluster of 2 ECS hosts distributed across two Availability Zones (min: 2, max: 2, desired: 2).\n\nIt is ***not*** set up to scale automatically based on any policies (CPU, network, time of day, etc.).\n\nIf you would like to configure policy or time-based automatic scaling, you can add the [ScalingPolicy](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-policy.html) property to the AutoScalingGroup deployed in [infrastructure/ecs-cluster.yaml](infrastructure/ecs-cluster.yaml#L69).\n\nAs well as configuring Auto Scaling for the ECS hosts (your pool of compute), you can also configure scaling each individual ECS service. This can be useful if you want to run more instances of each container/task depending on the load or time of day (or a custom CloudWatch metric). To do this, you need to create [AWS::ApplicationAutoScaling::ScalingPolicy](http://docs.aws.amazon.com/pt_br/AWSCloudFormation/latest/UserGuide/aws-resource-applicationautoscaling-scalingpolicy.html) within your service template.\n\n### Deploy multiple environments (e.g., dev, test, pre-production)\n\nDeploy another CloudFormation stack from the same set of templates to create a new environment. The stack name provided when deploying the stack is prefixed to all taggable resources (e.g. EC2 instances, VPCs, etc.) so you can distinguish the different environment resources in the AWS Management Console.\n\nTo distinguish between e.g. staging and production configurations, you will need to author multiple `master.yaml` files, each with the specific parameter values (e.g. `Host` or `PublicAlbAcmCertificate`) that address e.g. the specific DNS addresses to reach each stack's otherwise-nearly-identical resources.\n\n### Change the VPC or subnet IP ranges\n\nThis set of templates deploys the following network design:\n\n| Item | CIDR Range | Usable IPs | Description |\n| --- | --- | --- | --- |\n| VPC | 10.180.0.0/16 | 65,536 | The whole range used for the VPC and all subnets |\n| Public Subnet | 10.180.8.0/21 | 2,041 | The public subnet in the first Availability Zone |\n| Public Subnet | 10.180.16.0/21 | 2,041 | The public subnet in the second Availability Zone |\n| Private Subnet | 10.180.24.0/21 | 2,041 | The private subnet in the first Availability Zone |\n| Private Subnet | 10.180.32.0/21 | 2,041 | The private subnet in the second Availability Zone |\n\nYou can adjust the CIDR ranges used in this section of the [master.yaml](master.yaml) template:\n\n```yaml\nVPC:\n  Type: AWS::CloudFormation::Stack\n    Properties:\n      TemplateURL: !Sub ${TemplateLocation}/infrastructure/vpc.yaml\n      Parameters:\n        EnvironmentName:    !Ref AWS::StackName\n        VpcCIDR:            10.180.0.0/16\n        PublicSubnet1CIDR:  10.180.8.0/21\n        PublicSubnet2CIDR:  10.180.16.0/21\n        PrivateSubnet1CIDR: 10.180.24.0/21\n        PrivateSubnet2CIDR: 10.180.32.0/21\n```\n\n### Update an ECS service to a new Docker image version\n\nECS has the ability to perform rolling upgrades to your ECS services to minimize downtime during deployments. For more information, see [Updating a Service](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/update-service.html).\n\nTo update one of your services to a new version, adjust the `Image` parameter in the service template (in [services/*](services/) to point to the new version of your container image. For example, if `1.0.0` was currently deployed and you wanted to update to `1.1.0`, you could update it as follows:\n\n```yaml\nTaskDefinition:\n  Type: AWS::ECS::TaskDefinition\n  Properties:\n    ContainerDefinitions:\n      - Name: your-container\n        Image: registry.example.com/your-container:1.1.0\n```\n\nAfter you've updated the template, update the deployed CloudFormation stack; CloudFormation and ECS handle the rest.\n\nTo adjust the rollout parameters (min/max number of tasks/containers to keep in service at any time), you need to configure `DeploymentConfiguration` for the ECS service.\n\nFor example:\n\n```yaml\nService:\n  Type: AWS::ECS::Service\n    Properties:\n      ...\n      DesiredCount: 4\n      DeploymentConfiguration:\n        MaximumPercent: 200\n        MinimumHealthyPercent: 50\n```\n\n## Contributing\n\nPlease [create a new GitHub issue](https://github.com/hackoregon/hackoregon-aws-infrastructure/issues/new) for any feature requests, bugs, or documentation improvements.\n\nWhere possible, please also [submit a pull request](https://help.github.com/articles/creating-a-pull-request-from-a-fork/) for the change.\n\n## License\n\nCopyright 2011-2016 Amazon.com, Inc. or its affiliates. All Rights Reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\"). You may not use this file except in compliance with the License. A copy of the License is located at\n\n[http://aws.amazon.com/apache2.0/](http://aws.amazon.com/apache2.0/)\n\nor in the \"license\" file accompanying this file. This file is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhackoregon%2Fhackoregon-aws-infrastructure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhackoregon%2Fhackoregon-aws-infrastructure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhackoregon%2Fhackoregon-aws-infrastructure/lists"}