{"id":15556139,"url":"https://github.com/so0k/ecs-sample","last_synced_at":"2025-04-23T20:45:27.593Z","repository":{"id":146553890,"uuid":"61320083","full_name":"so0k/ecs-sample","owner":"so0k","description":"Sample for ECS demo","archived":false,"fork":false,"pushed_at":"2016-07-06T11:29:55.000Z","size":1107,"stargazers_count":12,"open_issues_count":8,"forks_count":6,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-18T05:53:31.163Z","etag":null,"topics":["docker","ecs-cluster","golang","mongodb","pubsub","redis"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/so0k.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-06-16T19:25:46.000Z","updated_at":"2025-02-10T12:32:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"4998b156-1407-4b12-90ed-6e4c2cc955f7","html_url":"https://github.com/so0k/ecs-sample","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/so0k%2Fecs-sample","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/so0k%2Fecs-sample/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/so0k%2Fecs-sample/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/so0k%2Fecs-sample/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/so0k","download_url":"https://codeload.github.com/so0k/ecs-sample/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250513380,"owners_count":21443200,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker","ecs-cluster","golang","mongodb","pubsub","redis"],"created_at":"2024-10-02T15:12:09.183Z","updated_at":"2025-04-23T20:45:27.585Z","avatar_url":"https://github.com/so0k.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Overview\n\nIn this sample Repo we will use Docker to quickly and easily get started with and scale a Real-Time messaging app written in Golang.\n\n![Screenshot of app](docs/images/app_screenshot.png)\n\nAs a reference, a very lightweight app from the [Toptal: Going Real-Time with Redis Pub/Sub](https://www.toptal.com/go/going-real-time-with-redis-pubsub) article was used. The code has slightly been modified to better demonstrate Docker related concepts and is fully included in this repository.\n\nChanges to the original application:\n\n- Added additional logging\n- Added indicator which application container is serving the client\n- Added [exponential back-off](https://medium.com/@kelseyhightower/12-fractured-apps-1080c73d481c#.zbqpolxwo) as a best practice for applications running in the cloud.\n\nThe application allows users to upload images and see real time comments on those images. Clicking the image will show indicators where the image was clicked for every user. All this functionality was written by the Toptal developer.\n\nTo implement the above functionality, the following stack will be used:\n\n- AWS S3: To store the user-uploaded images.\n- MongoDB: As a Document Oriented Database keeping track of images stored on S3 and the comments of users.\n- Redis Pub/Sub: Redis as a Publish/Subscribe messaging system to propagate real time updates\n- App: the Golang application to serve the webpage and manage the websockets with client browsers\n- Nginx: As a load balancer to easily scale the application horizontally.\n\n**Note**: Nginx is used as a load balancer while running the full stack locally, in a production environment a more robust load balancer setup should be considered.\n\n*Note* IAM permissions for EC2, ECS and CloudFormation are required to follow along this guide.\n\nDevelopment environment:\n\n - Tested on OSX El Capitan with Bash\n - Install [Docker For Mac](https://beta.docker.com/)\n - Ensure you have a working AWS Account (we will do sample S3 setup as part of these instructions)\n - Download [jq](https://github.com/stedolan/jq/releases) to easily work with AWS resources from the CLI\n\n## Table of Contents:\n\n - [Setting up S3](#setting-up-s3) : Set up all necessary objects on AWS using the CLI.\n - [Getting to know Docker for Mac](#getting-to-know-docker-for-mac): Ensuring Docker is working as expected.\n - [Getting to know Redis](#getting-to-know-redis): Using Docker to play with Redis - no installs required!\n - [Getting to know MongoDB](#getting-to-know-mongodb): Using Docker to play with MongoDB - no installs required!\n - [Playing with the full application stack](#playing-with-the-full-application-stack): `git clone` + `make` + `docker-compose` to stand up the full app locally.\n - [Understanding Container Distribution](#understanding-container-distribution): How do we take containers to the cloud?\n - [Amazon ECS Introduction](#amazon-ecs-introduction): High level overview of ECS (To be completed)\n - [Deploying to Amazon ECS](#deploying-to-amazon-ecs): Using AWS CLI to deploy full app to ECS (To be completed)\n\n## Setting up S3\n\nInstall AWS-CLI (You will need access keys to use the cli)\n```\npip install awscli\naws configure\n```\n\nEnsure jq is working properly:\n```console\njq --version\n```\n\nExpected Output (similar to this):\n```console\njq-1.5\n```\n\nCreate an account to give S3 access to the Application (Don't use root account)\n```console\naws iam create-user --user-name sample_app\n```\n\nCreate Access Key and save to `.env` file:\n```console\naws iam create-access-key --user-name sample_app | jq -r '\"AWS_ACCESS_KEY_ID=\\(.AccessKey.AccessKeyId)\",\"AWS_SECRET_ACCESS_KEY=\\(.AccessKey.SecretAccessKey)\"' \u003e\u003e .env\n```\n\nCreate S3 Bucket (You will need to change instructions according to your bucket name, samples here use `ecs-sample`)\n```console\naws s3 mb s3://ecs-sample --region ap-southeast-1\n```\n\nAdd your S3 bucket name to your `.env` file:\n```console\necho \"S3_BUCKET_NAME=ecs-sample\" \u003e\u003e .env\n```\n\nCreate Policy Document for S3 Bucket\n```console\ncat - \u003c\u003c EOF \u003e SampleAppS3Policy.json\n{\n    \"Version\": \"2012-10-17\",\n    \"Statement\": [\n        {\n            \"Effect\": \"Allow\",\n            \"Action\": \"s3:*\",\n            \"Resource\": [\n                \"arn:aws:s3:::ecs-sample/\",\n                \"arn:aws:s3:::ecs-sample/*\"\n            ]\n        }\n    ]\n}\nEOF\n```\n\nUpload Policy Document\n```console\npolicy_arn=`aws iam create-policy --policy-name MyAppS3Access --description \"Give S3 Access to ecs-sample bucket\" --policy-document file://SampleAppS3Policy.json | jq -r '.Policy.Arn'`\n```\n\nAttach Policy Document to `sample_app` user\n```console\naws iam attach-user-policy --user-name sample_app --policy-arn $policy_arn\n```\n\n**Note**: CloudFormation should be considered to automate the above steps.\n\n## Getting to know Docker for Mac\n\nOnce Docker for Mac has been installed and is running, confirm everything is installed correctly:\n\n```console\ndocker version\n```\n\nExpected output (similar to this)\n```console\nClient:\n Version:      1.11.2\n API version:  1.23\n Go version:   go1.5.4\n Git commit:   b9f10c9\n Built:        Wed Jun  1 21:20:08 2016\n OS/Arch:      darwin/amd64\n\nServer:\n Version:      1.11.2\n API version:  1.23\n Go version:   go1.5.4\n Git commit:   56888bf\n Built:        Mon Jun  6 23:57:32 2016\n OS/Arch:      linux/amd64\n```\n\n```console\ndocker-compose version\n```\n\nExpected output (similar to this)\n```console\ndocker-compose version 1.7.1, build 0a9ab35\ndocker-py version: 1.8.1\nCPython version: 2.7.9\nOpenSSL version: OpenSSL 1.0.1j 15 Oct 2014\n```\n\ndocker \"Hello world!\"\n\n```console\ndocker run hello-world\n```\n\nExpected output:\n```console\n...\na9d36faac0fe: Pull complete\nDigest: sha256:e52be8ffeeb1f374f440893189cd32f44cb166650e7ab185fa7735b7dc48d619\nStatus: Downloaded newer image for hello-world:latest\n\nHello from Docker.\nThis message shows that your installation appears to be working correctly.\n\nTo generate this message, Docker took the following steps:\n 1. The Docker client contacted the Docker daemon.\n 2. The Docker daemon pulled the \"hello-world\" image from the Docker Hub.\n 3. The Docker daemon created a new container from that image which runs the\n    executable that produces the output you are currently reading.\n 4. The Docker daemon streamed that output to the Docker client, which sent it\n    to your terminal.\n\nTo try something more ambitious, you can run an Ubuntu container with:\n $ docker run -it ubuntu bash\n\nShare images, automate workflows, and more with a free Docker Hub account:\n https://hub.docker.com\n\nFor more examples and ideas, visit:\n https://docs.docker.com/engine/userguide/\n```\n\nAt this point, you may read the [userguide](https://docs.docker.com/engine/userguide/) linked above. Concepts will be explained as they are encountered in this guide as well.\n\n## Getting to know Redis\n\nOnce Docker is installed, you will never have to install packages on your machine to play with interesting technology again. You can simple run the service in a container and remove every trace of it when done.\n\nFirst, lets spin up a Daemonized (`-d`) redis container (named `redis-test`):\n```console\ndocker run -d --name redis-test redis:3.2.0-alpine\n```\n\nVerify the container is running:\n```console\ndocker ps\n```\n\nTo play with this redis, we need the `redis-cli`, but we do not need to install it on our machine as `redis-cli` is bundled in the redis container!\n\nGet a shell (`-it`) in a 2nd redis container linked (`--link`) to the first:\n```console\ndocker run -it --rm --link redis-test redis:3.2.0-alpine /bin/sh\n```\n\nFrom within this container, connect to redis server (`-h redis-test`):\n```console\nredis-cli -h redis-test\n```\n\nTest Redis\n```console\nSET lives 9\nINCR lives\nGET lives\n```\n\nLet's play with Pub/Sub features of Redis:\n```console\nSUBSCRIBE channel\n```\n\nLaunch a 2nd redis container (re-use the exact same command to launch the container from above in a separate terminal)\n```console\nredis-cli -h redis-test\n```\n\nPublish data from 2nd container to `channel`\n```console\nPUBLISH channel \"hello from container2\"\n```\n\nYou should see the message broadcasted to all subscribed clients. Notice that once a connection is in subscription mode, you can no longer use it to send messages on. To both send and receive, 2 separate connections to the redis server are required.\n\n## Getting to know MongoDB\n\nVery similar to the Redis experiment above, quickly launch a MongoDB server:\n\nLaunch a Daemonised Mongo container:\n```console\ndocker run -d --name mongo-test mongo:3.2.6\n```\n\nLaunch a container to play with the mongo shell:\n```console\ndocker run -it --rm --link mongo-test mongo:3.2.6 /bin/bash\n```\n\nConnect via Mongo shell\n```console\nmongo mongo-test:27017\n```\n\nInsert sample documents\n```javascript\ndb.messages.insert(\n{\n    \"message\": \"hello\",\n    \"sender\": \"me\"\n})\n\ndb.messages.insert(\n{\n    \"message\": \"world\",\n    \"sender\":\"you\"\n})\n```\n\nSelect sample messages\n```javascript\ndb.messages.find()\n```\n\nSelect sample messages with a conditions document\n```javsacript\ndb.messages.find( { \"sender\": \"you\" })\n```\n\nCreate an ascending index on `sender` field of the messages collection\n```javascript\ndb.messages.createIndex({\"sender\": 1})\n```\n\n## Playing with the full application stack\n\nThe application is written in Golang. all dependencies have been vendored in with `Godeps`. However, to play with the application, golang does not have to be installed locally. Everything is handled through Docker.\n\nLet's first clean up the `redis-test` and `mongo-test` containers:\n```console\ndocker stop redis-test mongo-test \u0026\u0026 docker rm redis-test mongo-test\n```\n\nClone the application:\n```console\ngit clone https://github.com/so0k/ecs-sample.git\n```\n\n**Note**: Once you are required to develop further, setting up a local golang environment and cloning the application under the correct path is still easy and possible.\n\nBuild the application (using Docker)\n```console\nmake\n```\n**Note**: This makefile is inspired by [Nicola Paolucci's article](https://developer.atlassian.com/blog/2015/07/osx-static-golang-binaries-with-docker/).\n\nThe full application stack is defined in a declarative [docker-compose.yaml](docker-compose.yaml) file at the root of this repository.\n\nThe Environment configuration for our application is stored in the `.env` file we have incrementally been creating in the above setup steps. Docker Compose will pass all these parameters from the `.env` file to our application via ENVIRONMENT VARIABLES.\n\nTwo parameters are still missing, add these as follows:\n```console\necho \"MONGO_URL=mongodb://mongo/ecs-sample\" \u003e\u003e .env\necho \"REDIS_URL=redis://redis\" \u003e\u003e .env\n```\n*Note*: We have defined the MongoDB hostname as `mongo` and the Redis hostname as `redis` in the docker-compose file.\n\nLet's `watch` running containers with the following command in a separate terminal:\n```console\nwatch -n 1 \"docker ps --format='table{{.Image}}\\t{{.Names}}\\t{{.Status}}\\t{{.Ports}}\\t{{.ID}}'\"\n```\n\nWe are now ready to stand up the application stack:\n```console\ndocker-compose up -d\n```\n\nOnce all containers are running, you should be able to open [localhost:80](http://localhost:80/)\n\nTry to scale the application:\n```console\ndocker-compose scale app=5\n```\n\nOpening multiple browsers should demonstrate that client sessions are load balanced to separate application servers behind the load balancers and that all real-time events are propagated across the cluster.\n\nGet a shell on one of the running application containers:\n```console\ndocker exec -it ecssample_app_1 /bin/sh\n```\n\nReview the DNS records published by the Docker Engine:\n```console\nnslookup apps\n```\n\nWe can ping `mongo` and `redis` hosts from app container:\n```console\nping mongo\nping redis\n```\n\nOur `mongo` and `redis` containers are isolated from the load balancer:\n\nGet a shell on the running load balancer.\n```console\ndocker exec -it lb /bin/sh\n```\n\nWe can ping apps:\n```console\nping apps\n```\n\nWe can not ping `mongo` or `redis` hosts\n```console\nping mongo\nping redis\n```\n\nClean up\n```console\ndocker-compose stop\ndocker-compose rm\n```\n\nNotes:\n\n- The application uses DNS round robin for the nginx load balancer to test scaling locally\n- A `so0k/ecs-sample:dev` container is available to play with the application source code\n\nRun the Dev container as follows:\n```console\ndocker run -it --name ecs-sample-dev -v $PWD:/go/src/github.com/so0k/ecs-sample so0k/ecs-sample:dev\n```\n\n## Understanding Container Distribution\n\nThe container images built as part of this demonstration are currently only available on our host. An important component of the container\neco-system is having the ability to ship these images to different hosts.\n\nSimilar to the concept of Software Repositories, container image repositories are designed for the purpose of delivering container\nimages to hosts. In the Docker ecosystem, distribution is managed through a Registry service.  Several Registry service offerings are available:\n\n1. *Docker Hub*: This is the public Registry provided by Docker and the default registry used by every Docker client.\n   The `mongo`, `redis` and `nginx` images we ran earlier were all obtained from the Docker Hub.\n   Public Repositories on the Docker Hub are free (Similar to Public Repositories on GitHub).\n\n2. *Self-Hosted Registry*: An open source version of the [Registry](http://github.com/docker/distribution) is available on GitHub.\n   This allows anyone to host Docker Images privately, with the overhead of configuring and maintaining the service.\n\n3. *Amazon ECR*: [Amazon ECR](http://aws.amazon.com/ecr/) is integrated with Amazon ECS and eliminates the need to operate\n   your own container repositories or  worry about scaling the underlying infrastructure.\n   Integration with IAM provides resource-level control of each repository. You pay only for the amount of data you store\n   in your repositories and data transferred to the internet.\n\nFor this sample application, a public repository on the Docker Hub was used, following these steps:\n\n1.   Create a Docker Hub account by [signing up](https://hub.docker.com/). Similar to GitHub, credentials were not required to pull images, however they are required to push images.\n2.   Provide your docker client with your account credentials:\n\n     ```console\n     docker login\n     ```\n\n3.   Review the repository and image name in the Makefile provided with this repository (change to match your Docker Hub  account and rebuild image if needed)\n4.   You may use the `make push` target to tag and push the container image to the Docker Hub:\n\n     ```console\n     make push\n     ```\n\n*Note* Changing the Repository and Image name in the Makefile will also require you to revise the `docker-compose.yaml`.\nThe changes required to this file are not covered in the current version of this guide, PR's are welcome.\n\n## Docker 1.12 Introduction\n\n### Using Docker For Mac:\n\n```\ndocker swarm init\ndocker service create ...\n```\n\n### Deploying 3 node cluster on EC2\n\n#### Provision the AWS infrastructure Using terraform\n\nCreate a `terraform.tfvars` file at the root of the directory with you AWS keys\n\nSample file:\n```\naccess_key = \"\u003cSAMPLE\u003e\"\nsecret_key = \"\u003cSAMPLE\u003e\"\n\npublic_key_path = \"~/.ssh/id_rsa.pub\"\nkey_name = \"ecs-sample\"\n```\n\nReview the infrastructure defined in `docker-cluster.tf` of this repository:\n```\nterraform plan\n```\n\nCreate the cluster on AWS:\n```\nterraform apply\n```\n\nAfter it completes, it should have returned a comma separated list of the nodes.\n\nYou may extract this list again from the local terraform state as follows:\n\n```\nterraform output nodes\n```\n\n#### Create the Docker 1.12 Cluster\n\nssh to the nodes and let the engines form a swarm\n\n```\nssh ubuntu@\u003cfirst-node-ip\u003e\nsudo -i\ndocker swarm init\n```\nNote private IP of Leader node\n\nRun the node visualizer\n```\ndocker run -it -d -p 3000:3000 -e HOST=\u003cpub-ip\u003e -e PORT=3000 -v /var/run/docker.sock:/var/run/docker.sock manomarks/visualizer\n```\n\n```\nssh ubuntu@\u003cother-nodes\u003e\nsudo docker swarm join \u003cleader-priv-ip\u003e:2377\n```\n\nNow, from the swarm leader:\n\n```\ndocker node ls\n```\n\n#### Play with Docker Service concept\n\nFrom the master node:\n\n```\ndocker service create --replicas 1 --name helloworld alpine ping docker.com\n```\n\n```\ndocker service ls\ndocker service inspect --pretty helloworld\ndocker service scale helloworld=5\n```\n\nTo see which nodes are running the tasks:\n```\ndocker service tasks helloworld\n```\n\nTo delete this service\n```\ndocker service rm helloworld\n```\n\n\nSee also:\n\n* `docker service create --mode=global`: services required on every node\n* `docker service create --constraint com.example.storage=ssd`: assumes `docker daemon --label com.example.storage=ssd`\n* [Bring node down for maintenance](https://github.com/mikegcoleman/labs/tree/master/dockercon-us/docker-orchestration#step-34---bring-a-node-down-for-maintenance)\n\n#### Create Distributed Application Bundle (DAB)\n\n[See docker/experimental/dab](https://docker.com/dab)\n\n```\ndocker-compose bundle\n```\n*Note*: the .env file is stored within the bundle (this may expose certain secrets)\n\n```\ndocker deploy ecssample\n```\n\n[See DockerCon Demo](https://youtu.be/vE1iDPx6-Ok?t=6405)\n```\ndocker service update -p\n```\n\nIdeally we'd use Instance Roles in our Terraform plan for the EC2 Instances created.\n\n## Amazon ECS Introduction\n\nAmazon EC2 Container Service (ECS) is a container management service which allows you to run containers\non a managed cluster of Amazon EC2 Instances. This eliminates the need of installing, operating and\nscaling your own cluster management infrastructure.\n\nAmazon ECS ties in with other Amazon Services such as IAM, Elastic Load Balancer, ...\n\nTo set up a sample cluster we will  use the [Amazon ECS CLI](https://github.com/aws/amazon-ecs-cli) Tool, see the instructions below.\nTo set up a cluster follow the [Setting Up](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/get-set-up-for-amazon-ecs.html) instructions on AWS\n\n### Components\n\n- Cluster: EC2 Instances grouped together to run containers.\n  Each instance runs the [open source ecs-agent](https://github.com/aws/amazon-ecs-agent) which starts containers on behalf of Amazon ECS.\n- Task Definitions: Task Definitions specify the configuration of a group of containers which should be ran on the same host. The concept is similar to the concept of Container Pods.\n- Schedulers: The Schedulers used by Amazon ECS define how containers are ran\n\n## Deploying to Amazon\n\nWe will be deploying our application containers to an Amazon ECS cluster to easily scale and manage the running instances using containers in a way identical to the stack used for local development. However, for the services (Redis, MongoDB) our application relies on, we will take a slightly different approach.\n\nFor Redis we will take advantage of the managed service provided through [Amazon ElastiCache](https://aws.amazon.com/elasticache/) and for MongoDB we will deploy a dedicated cluster outside of Amazon ECS.\n\nAs an alternative to the self-service approach required for MongoDB, we could use the fully managed [Amazon DynamoDB](http://aws.amazon.com/dynamodb/) NoSQL database service.\n\nTo migrate from MongoDB to DynamoDB we will need to change the implementation of the persistance layer of our application.\n\nFor development purposes we may follow [these AWS guidelines](https://aws.amazon.com/blogs/aws/dynamodb-local-for-desktop-development/) to build a DynamoDB Docker Image, as a reference see [this Dockerfile](https://hub.docker.com/r/deangiberson/aws-dynamodb-local/~/dockerfile/).\n\nOnce we have an up to date docker DynamoDB image we will also need to make few changes to our `docker-compose.yaml` file to stand up the new stack.\n\n### Deploying the NoSQL Document store\n\nTo keep this sample simple, we will use the [MongoDB on AWS Quick Start Guide](http://docs.aws.amazon.com/quickstart/latest/mongodb/overview.html) and create the stack using [Cloudformation from the AWS CLI](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/using-cfn-cli-creating-stack.html). The configurations used are not recommended for a production deploy but are acceptable for demo purposes.\n\nVerify at least 1 key-pair exists to work with EC2 instances:\n\n```\naws ec2 describe-key-pairs\n```\n\nIf you do not have any key-pair, follow [These instructions](http://docs.aws.amazon.com/cli/latest/reference/ec2/import-key-pair.html) to import a new key pair into EC2.\n\n**Note**: The Quickstart template creates a NAT Instance in a DMZ subnet while keeping the MongoDB Instances in a private subnet. As of December 2015, AWS offers a [Managed EC2 NAT gateway](https://aws.amazon.com/blogs/aws/new-managed-nat-network-address-translation-gateway-for-aws/) and it is no longer required to manage a NAT instance by yourself.\n\nCreate a Parameter file for the MongoDb Cloudformation Template:\n\n**Note**: Don't forget to update the name of the key pair\n```\ncat \u003c\u003cEOF \u003e mongo-stack-parameters.json\n[\n  {\n    \"ParameterKey\": \"AvailabilityZone0\",\n    \"ParameterValue\":\"ap-southeast-1a\"\n  },\n  {\n    \"ParameterKey\": \"AvailabilityZone1\",\n    \"ParameterValue\":\"ap-southeast-1b\"\n  },\n  {\n    \"ParameterKey\": \"AvailabilityZone2\",\n    \"ParameterValue\":\"ap-southeast-1b\"\n  },\n  {\n    \"ParameterKey\": \"ClusterReplicaSetCount\",\n    \"ParameterValue\":\"1\"\n  },\n  {\n    \"ParameterKey\": \"ClusterShardCount\",\n    \"ParameterValue\":\"0\"\n  },\n  {\n    \"ParameterKey\": \"KeyName\",\n    \"ParameterValue\":\"your-key\"\n  },\n  {\n    \"ParameterKey\": \"MongoDBVersion\",\n    \"ParameterValue\":\"3.0\"\n  },\n  {\n    \"ParameterKey\": \"NodeInstanceType\",\n    \"ParameterValue\":\"m3.medium\"\n  }\n]\nEOF\n```\n\nCreate the MongoDB stack in a new VPC using the QuickStart Template:\n```\naws cloudformation create-stack \\\n  --stack-name \"mongo-quickstart\" \\\n  --capabilities CAPABILITY_IAM \\\n  --template-url https://s3.amazonaws.com/quickstart-reference/mongodb/latest/templates/MongoDB-VPC.template \\\n  --parameters file://mongo-stack-parameters.json\n```\n\nTo review the status of each Stack resource using the CLI:\n```\naws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq '.StackResourceSummaries[] | {Type: .ResourceType, Status: .ResourceStatus}'\n```\n\nTo Review the status of the overall stack creation process using the CLI:\n```\naws cloudformation describe-stacks --stack-name \"mongo-quickstart\" | jq -r '.Stacks[0].StackStatus'\n```\n\nGet the Private IP of the MongoDB Instance (this command may differ for you if you changed the parameters above):\n\n```\nMONGO_INSTANCE=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"PrimaryReplicaNode0NodeInstanceGP2\") | .PhysicalResourceId '`\nMONGO_INSTANCE_IP=`aws ec2 describe-instances --instance-ids $MONGO_INSTANCE | jq -r '.Reservations[].Instances[].PrivateIpAddress'`\n\n```\n\nGet the Public IP of the NAT Instance (we may use this as a jump host while troubleshooting):\n```\nNAT_INSTANCE=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"NATInstance\") | .PhysicalResourceId '`\nNAT_INSTANCE_IP=`aws ec2 describe-instances --instance-ids $NAT_INSTANCE | jq -r '.Reservations[].Instances[].PublicIpAddress'`\nssh ec2-user@$NAT_INSTANCE_IP\n```\n\n### Creating the Amazon ElastiCache Service\n\nGet the VPC and Private Subnet of the Mongo Cluster:\n```\nVPC=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"VPC\") | .PhysicalResourceId '`\n\nPRIVATE_SUBNET=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"PrimaryNodeSubnet\") | .PhysicalResourceId '`\n```\n\nCreate the ElastiCache subnet into the existing private subnet:\n```\naws elasticache create-cache-subnet-group --cache-subnet-group-name \"ecs-sample-cache\" --cache-subnet-group-description \"ecs sample\" --subnet-ids $PRIVATE_SUBNET\n```\n\nGet GroupId of Default Security Group:\n```\nDEFAULT_SG=`aws ec2 describe-security-groups --filters Name=vpc-id,Values=$VPC | jq '.SecurityGroups[] | select(.GroupName == \"default\") | .GroupId'`\n```\n\n```\naws elasticache create-cache-cluster \\\n  --engine redis \\\n  --cache-node-type cache.t2.micro \\\n  --num-cache-nodes 1 \\\n  --cache-subnet-group-name ecs-sample-cache \\\n  --cache-group-ids $DEFAULT_SG\n  --cache-cluster-id ecs-sample\n```\n\nFinding endpoints (returns endpoint for first Node in Cluster)\n```\naws elasticache describe-cache-clusters \\\n    --cache-cluster-id ecs-sample \\\n    --show-cache-node-info | jq -r .CacheClusters[].CacheNodes[0].Endpoint.Address\n```\n\n### Setting up the ECS Cluster in the existing DMZ Subnet\n\nAmazon ECS clusters are fully customizable through CloudFormation Templates, refer to the [ECS CloudFormation Snippet](http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/quickref-ecs.html) in the Amazon Docs as a reference for creating a production ready cluster.\n\nFor this sample, however, we will use the `ecs-cli` tool as a means to quickly provision a Sample ECS cluster into the DMZ subnet of our Mongo stack.\nDownload the latest release of the `ecs-cli` tool with the following commands:\n\n```console\ncurl -Lo ecs-cli https://s3.amazonaws.com/amazon-ecs-cli/ecs-cli-darwin-amd64-latest\nchmod +x ./ecs-cli\nln -s $PWD/ecs-cli /usr/local/bin/ecs-cli\n```\n\nAlthough the `ecs-cli` tool reads AWS credentials from the same `~/.aws/credentials` file as the `aws cli` tool (which we used in the section setting up S3), some additional configuration is required (cluster name and region need to be defined) at the time of writing:\n\n```\necs-cli configure --cluster \"ecs-sample\" -r ap-southeast-1\n```\n\nGet the VPC and DMZ Subnet of the Mongo Cluster:\n```\nVPC=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"VPC\") | .PhysicalResourceId '`\n\nDMZ_SUBNET=`aws cloudformation list-stack-resources --stack-name \"mongo-quickstart\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"DMZSubnet\") | .PhysicalResourceId '`\n```\n\n**NOTE**: Ideally 2 subnets across 2 availability zones are used instead.\n\nFinally, create a sample `ecs-cluster` in the DMZ Subnet of the Mongo Cluster with the following commands:\n```\necs-cli up --keypair \u003ckey-pair-name\u003e --capability-iam \\\n  --size 2 \\\n  --vpc \"$VPC\"\n  --subnets \"$DMZ_SUBNET\" \\\n  --image-id \"ami-cf03d2ac\" \\\n  --instance-type t2.micro\n```\n\nWhere the following parameters were used:\n\n- `--capability-iam`:  Indicates we aknowledge the ecs-cli tool may create IAM resources\n- `--size 2` :  Indicates we want 2 EC2 Instances (default = 1)\n- `--vpc`:  Takes an existing VPC (the same VPC created earlier for our MongoDB Cluster)\n- `--subnets`: Takes a comma separated list of subnets (The Public subnet of our Mongo VPC in this example)\n- `--image-id` :  Should specify the up to date Id for the ECS-optimized AMI of the required region, revise based on [updated list here](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html?adbsc=docs_20160617_62622486\u0026adbid=743951143777120256\u0026adbpl=tw\u0026adbpr=66780587).\n- `--instance-type` : `t2.micro` equals to 1 vCPU and 1 GB of Memory per instance, which should be sufficient for this demo.\n\nOnce creation of the cluster completed, we may test connectivity from any of the EC2 Instances to our MongoDB cluster as follows:\n\nGet the Public IP of such an instance:\n```\nECS_ASG=`aws cloudformation list-stack-resources --stack-name \"amazon-ecs-cli-setup-ecs-sample\" | jq -r '.StackResourceSummaries[] | select(.LogicalResourceId == \"EcsInstanceAsg\") | .PhysicalResourceId'`\n\nECS_INSTANCE1=`aws autoscaling describe-auto-scaling-groups --auto-scaling-group-names $ECS_ASG | jq -r .AutoScalingGroups[].Instances[0].InstanceId`\n\nECS_INSTANCE1_IP=`aws ec2 describe-instances --instance-ids $ECS_INSTANCE1 | jq -r '.Reservations[].Instances[].PublicIpAddress'`\n```\n**TO BE COMPLETED**\nWe will need to modify the ECS Security Group to allow SSH access.\n```\n#TO BE COMPLETED - done through AWS Console for now...\naws ec2 describe-security-groups...\n```\n\nConnect to the instance (confirm the identity when prompted):\n```\nssh ec2-user@$ECS_INSTANCE1_IP\n```\n\nGet a shell into a Mongo container (this will pull the image first)\n```\ndocker run -it --rm mongo:3.2.6 /bin/bash\n```\n\nFrom this shell, test connectivity to the MongoDB instance:\n```\nmongo \u003cprivate-ip-of-mongo-instance\u003e:27017\n```\n\n### Creating the Task Definition\n\nTask definition under construction, see `ecs-sample-app.json`\n\nUsing AWS CLI to deploy full app to ECS - To be completed.\n\n### Expose Application through ELB\n\n**Note**: Enable Websockets / Sticky Sessions on ELB...\n\n## Conclusion\n\nDocker really simplified getting started with a new technology stack with minimum setup of the local machine required.\n\nAWS ECS provides a robust infrastructure to run Docker Containers at scale in production.\n\nImages built are minimal and share layers where possible\n\nImage Sizes:\n```\nREPOSITORY          TAG                 CREATED             SIZE\necssample_lb        latest              2 hours ago         182.8 MB\nso0k/ecs-sample     dev                 2 hours ago         821.6 MB\nso0k/ecs-sample     1.0.0               3 hours ago         14.33 MB\nredis               3.2.0-alpine        7 days ago          29.07 MB\nmongo               3.2.6               3 weeks ago         313.1 MB\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fso0k%2Fecs-sample","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fso0k%2Fecs-sample","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fso0k%2Fecs-sample/lists"}