{"id":16420764,"url":"https://github.com/andrew-chen-wang/cookiecutter-django-ecs-github","last_synced_at":"2025-03-23T07:31:59.130Z","repository":{"id":103625030,"uuid":"255405202","full_name":"Andrew-Chen-Wang/cookiecutter-django-ecs-github","owner":"Andrew-Chen-Wang","description":"Complete Walkthrough: Blue/Green Deployment to AWS ECS using Cookiecutter-Django using GitHub actions","archived":false,"fork":false,"pushed_at":"2020-12-13T23:57:33.000Z","size":209,"stargazers_count":40,"open_issues_count":2,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-18T18:45:45.175Z","etag":null,"topics":["aws","cookiecutter-django","deployment","django","ecs"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Andrew-Chen-Wang.png","metadata":{"files":{"readme":"README.rst","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-04-13T18:05:10.000Z","updated_at":"2024-11-05T17:28:43.000Z","dependencies_parsed_at":"2023-05-24T01:15:21.227Z","dependency_job_id":null,"html_url":"https://github.com/Andrew-Chen-Wang/cookiecutter-django-ecs-github","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andrew-Chen-Wang%2Fcookiecutter-django-ecs-github","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andrew-Chen-Wang%2Fcookiecutter-django-ecs-github/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andrew-Chen-Wang%2Fcookiecutter-django-ecs-github/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Andrew-Chen-Wang%2Fcookiecutter-django-ecs-github/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Andrew-Chen-Wang","download_url":"https://codeload.github.com/Andrew-Chen-Wang/cookiecutter-django-ecs-github/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245071912,"owners_count":20556352,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","cookiecutter-django","deployment","django","ecs"],"created_at":"2024-10-11T07:29:07.671Z","updated_at":"2025-03-23T07:31:59.118Z","avatar_url":"https://github.com/Andrew-Chen-Wang.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"AWS ECS Deploy\n==============\n\nAWS ECS Deploy Blue/Green using GitHub Actions.\n\nSorry if this didn't work. I tried to do this all in ONE go, or one commit. ~ Andrew C. 29 May 2020. Ripperoni.\n\nEdit: It works! It only took so long... and 3 days of finally realizing it security groups is a thing...\nI'm a beginner, so start-ups, take advantage of this repository! ~ Andrew C. 10 June 2020.\n\nMuch appreciated code taken from awslabs: https://github.com/awslabs/ecs-nginx-reverse-proxy/tree/master/reverse-proxy\n\n.. image:: https://img.shields.io/badge/built%20with-Cookiecutter%20Django-ff69b4.svg\n     :target: https://github.com/pydanny/cookiecutter-django/\n     :alt: Built with Cookiecutter Django\n\n:License: Apache Software License 2.0\n\nTable of contents:\n\n- How does this work?\n- Deployment Instructions\n- Cleanup\n- The Caveats in THIS EXAMPLE (easily avoidable)\n- Initial Cookiecutter Generation\n- Minimal IAM Credentials for ECS\n- Minimal IAM Credentials for Deployment\n- FAQ\n\nHow does this work?\n-------------------\n\nThe deployment is done through Travis CI which does a sorta webhook to a\nGitHub action which will use several AWS GH Actions to finally deploy\nyour application. TODO Actually create that webhook.\n\nThe AWS services that we'll be using are: CodeDeploy, ECS + ECR, Parameter Store,\nIAM, Application Load Balancer. CodeDeploy is for deploying from the GH\naction. ECS + ECR is where our servers will be located. Parameter store will\nbe where we store our secrets/environment variables. IAM is for proper\nsecurity measures of the credentials given to the server AND to GitHub for deployment.\nYes, we need two IAM users, one for GH and the other for ECS. Finally, we need\nALB for proper Blue/Green deployment assuming you have more than one instance\nin your ECS cluster.\n\nI chose ECS because there were GitHub actions for this anyways. However, note\nthat this is much slower than simply using EC2. The current Travis configuration\nuses Docker, but my PR in cookiecutter-django should change the configuration\nfrom Docker based to completely Docker free, erasing 1.5 minutes of CI.\n\nWhy not use Fargate? Personally, I like EC2 over Fargate or Lambda since\nI use celery a lot. That's really it; I haven't used AWS much, nor have\nI actually deployed an AWS app until now.\n\nI've set up an nginx reverse proxy to do a lot of heavy weighted work and to minimize\nsecurity risks. Additionally, I've left out several key components like PostgreSQL\nand Redis. Those can easily be added in the `aws-task-definition.json`. Just look\nat how the rest of the GH action is setup to configure PostgreSQL Dockerfile in\n`compose/production/postgres`.\n\nLet's start the deployment process!\n\nDeployment Instructions\n-----------------------\n\nThe following details how to deploy this application stack to ECS.\nIf you need to also setup your database and cache, follow \n[issue #9](https://github.com/Andrew-Chen-Wang/cookiecutter-django-ecs-github/issues/9).\nYou may want to get the hang of the following first, specifically security groups in step 1.\nAlso note that I didn't write that in the tutorial because it's spare and\ncame from my memory from a couple months back. I didn't think it was detailed\nenough for this tutorial.\n\nIt's a lot of instructions since there are so many services:\n\n1. You must have an IAM user with the correct permissions that you can find at the\nbottom of this README. Make sure you copy the ones from the section labeled\n`Minimal IAM Credentials for Deployment`. You can create a new policy\nusing the JSON below during the Set Permissions section by pressing\n\"Attach existing policies directly\" and pressing \"Create Policy.\"\n\nAdd the IAM user's credentials to your repo's secrets\nin the repo's settings. The credentials' names MUST be `AWS_ACCESS_KEY_ID`\nand `AWS_SECRET_ACCESS_KEY`\n\n2. Security Groups - Exposing your ports\n\nIt took me 10 prolonged days to figure out my security groups were not properly\nconfigured for my ALB and ECS instances. So follow this carefully.\n\nCreate a security group called WebAccess with a description saying it's for ALB.\n\nThe inbound traffic should be of 4 rules:\n\n- Type HTTP, with custom source type on source 0.0.0.0/0\n- Type HTTP, with custom source type on source ::/0\n- Type HTTPS, with custom source type on source 0.0.0.0/0\n- Type HTTPS, with custom source type on source ::/0\n\nThe outbound rules should be left on default, but just in case:\n\n- Type All traffic, destination type Custom at 0.0.0.0/0\n\nCreate another security group. This is for your ECS instances.\nName it ECS-reverse-proxy (for another website, I'd recommend\nprefixing the name of this with your website name).\n\nThe inbound traffic should be 1 rule only:\n\n- Type All Traffic, with custom source type, and find your first security\n  group (it should say the name of the security group you just created).\n\nThe outbound traffic is the same as the first one.\n\n3. Create two ECR repositories to store your images by running the following:\n\n.. code-block:: shell\n\n    aws ecr create-repository --repository-name nginx-reverse-proxy --region us-east-2\n    aws ecr create-repository --repository-name django-app --region us-east-2\n\nReplace the region with your AWS region, and make sure you change it in the\n`.github/workflows/aws.yml` file, as well.\n\n4. Create an ECS cluster.\nReplace the values for `cluster` in the aws workflow\nwith your cluster name. The default is \"cookiecutter-django\".\nI guess you could just write that one and not need to change\nthe one in the GH action.\n\n- Choose EC2 Linux + Networking\n- I chose t2.medium for the instance type for enough memory.\n\n  - The task definition uses a limited amount of memory as celery\n    isn't a main priority here. As you expand, celery will take up\n    more memory and you'll have to increase the memory capacity for\n    the Django app, which means you'll have to use a different\n    instance type.\n\n- You can just have one instance since Blue/Green deployments\n  will provision a new instance and deregister the old one.\n\n  - That's the downfall about ECS. You can configure everything\n    in your Dockerfile, but it's a slow build and start time and\n    you wish the instance could just simply be updated...\n\n- I had a key pair from previous EC2 usage. You don't necessarily need it\n  but it could be helpful to have on in the future. Yes, you can configure\n  an ssh key pair in the future.\n- Create a new VPC.\n- Choose a subnet. Remember which subnet it is.\n- Use that second security group that I said was for your ECS instances!\n- The IAM role can be the one created by them called ecsInstanceRole.\n\n5. Grant a service trust relationship for newly created IAM role\n\nIn order to add our environment variables via our task definition, we must\nmake sure the IAM role (above, hopefully it was ecsInstanceRole)\ncan even do a task execution.\n\nGo to your newly created IAM role and click \"Trust relationships\"\n\nEdit the trust relationship so that, in the \"Service\" array, you add\n`ecs-tasks.amazonaws.com`\n\n5. Buy a website in Route 53.\n\nI bought a random website with a `.de` ending since that came out to be $8.\nMy website was `asdfasq.de`. Random, ey?\n\nThe more random the name and extension, the cheaper.\n\nChange allowed hosts in `config/settings/production.py` to your domain.\n\nChange every instance of asdfasq.de in `compose/production/ecs/nginx/nginx.conf`\nto your domain.\n\n6. Configure ACM for https for your domain.\n\nFind ACM (certificate manager) and add your domain and\nits www. format, as well.\n\n7. Create the ALB, or Application Load Balancer with ACM\n\nNOTE: I might be missing a step with the certificate manager. I deployed\na test website on EC2 as a standalone, and I might've done something to\nproperly configure the certificate. PLEASE open a PR/Patch if I'm missing it.\n\nGo to the EC2 page. Find the Load Balancers section and create a new balancer.\n\n- Name your load balancer something like... Joe.\n- Add a new listener with HTTPS. The port should autofill itself to be 443.\n  Click next.\n- Your VPC and subnets should be the same as the ones you\n  SHOULD'VE WRITTEN DOWN in step 3 when creating your cluster.\n- I'm seeing my website and certificate. If you're not, then look online\n  for how to do that and open a PR.\n- Your security group is the first one you created in step 2.\n- Configure routing:\n\n  - Select new target group\n  - Name it something\n  - The protocol should be HTTP.\n  - Leave health check on default.\n\n- Don't register any instance.\n- Finally, create it.\n\n8. Add your load balancer to your hosted zone\n\nGo back to Route 53. Go to your hosted zone and add 2 A record\nsets. Choose yes for use alias. Find your load balancer.\n\nThe difference between each record set is that the first one\nfor name can be left blank while the other one should have www.\nThis is also how you can have multiple ECS clusters for different\napplications (i.e. with subdomains).\n\n9. Create a task definition.\n\nGo to the `aws-task-definition.json` file and copy its contents.\n\nIn the ECS dashboard, create a new task definition. Scroll to the\nbottom until you find \"configure via JSON.\" Paste the contents.\n\n10. Create an ECS service.\n\nAfter you finished creating your cluster, you should arrive in the service\ntab. Create a service.\n\n- Configure Service\n\n  - Launch type is obviously EC2\n  - Skip the Task Definition section.\n  - Choose your cluster if it's not the one you created.\n  - Enter a service name\n\n    - default in workflow is cookiecutter-django-service.\n    - If you use the default name, then you don't need to\n      change the one in the GH action.\n\n  - Number of tasks is 1\n  - The deployments section!\n\n    - Deployment type: Blue/Green\n\n      - I explained up top why I chose this one.\n      - Gist of it: CodeDeploy + Websockets + Slow shifting of Traffic.\n      - Deployment configuration: ECS Linear 10 Percent Every 1 Minute\n      - Service role for CodeDeploy: This is the IAM role that you should\n        have for your ECS instances. You can find my configuration down below\n        in the IAM role configuration sections with the one labeled `ECS`\n\n  - The service role for CodeDeploy should be the same one you created in step 1.\n    It should also, probably, be the only one in that dropdown.\n\n- Configure Network\n\n  - Choose application load balancer\n  - Health check grace period should be 15 seconds. This option is above the \"choose ALB.\"\n  - For Service IAM Role, I chose AWSServiceRoleForECS. Idk if that'll appear for you though.\n  - Select your load balancer\n  - Container to Load Balance:\n\n    - Make sure the container name and port is nginx:80\n    - Then press `Add to Load Balancer`\n\n      - Disable test listener\n\n  - Choose the target groups you made when making your ALB\n    for Target Group 1 and create a second target group.\n  - Service discovery\n\n    - Enable it since you've got a website\n    - Create a new, verbose private namespace.\n\n      - You want something unique... like cookiecutter-django-namespace1\n      - The namespace name can just be left as local\n\n    - The cluster VPC should be the one you had all along.\n\n      - Enable ECS task health propagation\n      - DNS records for service discovery should have the\n        container with nginx and TTL be 60 seconds.\n\n- Autoscaling policy. I didn't touch it and just said \"Do not adjust\".\n  You can adjust it later. (I honestly have no idea myself. You shouldn't\n  need to worry about it yet anyways).\n- Review and press that shiny blue button to create the service.\n\n11. Change your health target ports\n\nWhile you're creating the service, the review stage should show your\nnew target groups. If not, it's fine. The task will stop and regenerate.\n\nRight click on each target group and change the success codes at the bottom\nfrom `200` to `200,301` (you cannot add spaces).\n\n12. Let's add our environment variables.\n\nSearch up Systems Manager. Look for Parameter Store on the left side.\nYou'll need to add the parameters from `.envs/.production/template.django`.\n\nI've noted which ones you should add.\n\n13. Finally, commit to your repository and let your code be deployed.\n\nCleanup\n-------\n\nIf you tested this first on a random GitHub repository, here's how to clean\nthose resources up:\n\n- You should delete your created IAM roles or users for this test\n- Delete your GitHub secrets\n- Delete your AWS services. Here's a list, in order, of deletion:\n\n  - Application Load Balancer\n  - Target Groups\n  - EC2 Instances\n  - ECS Service\n  - ECS Cluster\n  - Task definition\n  - CodeDeploy application\n  - AWS Cloud Map namespace\n\nThe Caveats in THIS EXAMPLE (easily avoidable)\n----------------------------------------------\n\nI didn't want to make ANOTHER image just for Celery; instead, I just used:\n\n.. code-block:: shell\n\n    \u003e\u003e celery multi start -A config.celery_app worker beat\n\nI use Sentry to log all my Celery stuff, anyways, and it will come with\ncookiecutter-django if you opt-in.\n\nI also use RDS for PostgreSQL and ElastiCache for Redis. You don't HAVE to,\nbut that would mean you need to configure some more stuff in the\naws-task-definitions.json.\n\nIn the task definition, you can easily add the redis and PostgreSQL images. If you\nfollow the GitHub action of how I set up everything and how you can easily use the\nDockerfile in compose/production/postgres, then just follow how I did the Django app.\n\nInitial Cookiecutter Generation\n-------------------------------\n.. code-block:: shell\n\n    project_name [My Awesome Project]: AWS ECS Deploy\n    project_slug [aws_ecs_deploy]:\n    description [Behold My Awesome Project!]: AWS ECS Deploy (hopefully with Blue/Green) using GitHub Actions\n    author_name [Daniel Roy Greenfeld]: Andrew Chen Wang\n    domain_name [example.com]:\n    email [andrew-chen-wang@example.com]: acwangpython@gmail.com\n    version [0.1.0]:\n    Select open_source_license:\n    1 - MIT\n    2 - BSD\n    3 - GPLv3\n    4 - Apache Software License 2.0\n    5 - Not open source\n    Choose from 1, 2, 3, 4, 5 [1]: 4\n    timezone [UTC]:\n    windows [n]:\n    use_pycharm [n]:\n    use_docker [n]: y\n    Select postgresql_version:\n    1 - 11.3\n    2 - 10.8\n    3 - 9.6\n    4 - 9.5\n    5 - 9.4\n    Choose from 1, 2, 3, 4, 5 [1]:\n    Select js_task_runner:\n    1 - None\n    2 - Gulp\n    Choose from 1, 2 [1]:\n    Select cloud_provider:\n    1 - AWS\n    2 - GCP\n    3 - None\n    Choose from 1, 2, 3 [1]:\n    Select mail_service:\n    1 - Mailgun\n    2 - Amazon SES\n    3 - Mailjet\n    4 - Mandrill\n    5 - Postmark\n    6 - Sendgrid\n    7 - SendinBlue\n    8 - SparkPost\n    9 - Other SMTP\n    Choose from 1, 2, 3, 4, 5, 6, 7, 8, 9 [1]: 2\n    use_drf [n]:\n    custom_bootstrap_compilation [n]:\n    use_compressor [n]:\n    use_celery [n]: y\n    use_mailhog [n]:\n    use_sentry [n]:\n    use_whitenoise [n]:\n    use_heroku [n]:\n    Select ci_tool:\n    1 - None\n    2 - Travis\n    3 - Gitlab\n    Choose from 1, 2, 3 [1]:\n    keep_local_envs_in_vcs [y]:\n    debug [n]:\n\nMinimal IAM Credentials for ECS\n-------------------------------\n\nYou'll need these permissions for your ECS:\n- S3 Full Access\n\nMinimal IAM Credentials for Deployment\n--------------------------------------\n\nYou're probably thinking... wtf is with all these brackets.\nSecurity. Besides that, you can use asterisks for demonstration\nfor demonstration purposes.\n\nFor me, during testing, I just used FullAccess... Shh...\n\n.. code-block:: json\n\n    {\n       \"Version\":\"2012-10-17\",\n       \"Statement\":[\n          {\n             \"Sid\":\"RegisterTaskDefinition\",\n             \"Effect\":\"Allow\",\n             \"Action\":[\n                \"ecs:RegisterTaskDefinition\"\n             ],\n             \"Resource\":\"*\"\n          },\n          {\n             \"Sid\":\"PassRolesInTaskDefinition\",\n             \"Effect\":\"Allow\",\n             \"Action\":[\n                \"iam:PassRole\"\n             ],\n             \"Resource\":[\n                \"arn:aws:iam::\u003caws_account_id\u003e:role/\u003ctask_definition_task_role_name\u003e\",\n                \"arn:aws:iam::\u003caws_account_id\u003e:role/\u003ctask_definition_task_execution_role_name\u003e\"\n             ]\n          },\n          {\n             \"Sid\":\"DeployService\",\n             \"Effect\":\"Allow\",\n             \"Action\":[\n                \"ecs:DescribeServices\",\n                \"ecs:UpdateService\",\n                \"codedeploy:GetDeploymentGroup\",\n                \"codedeploy:CreateDeployment\",\n                \"codedeploy:GetDeployment\",\n                \"codedeploy:GetDeploymentConfig\",\n                \"codedeploy:RegisterApplicationRevision\"\n             ],\n             \"Resource\":[\n                \"arn:aws:ecs:\u003cregion\u003e:\u003caws_account_id\u003e:service/\u003ccluster_name\u003e/\u003cservice_name\u003e\",\n                \"arn:aws:codedeploy:\u003cregion\u003e:\u003caws_account_id\u003e:deploymentgroup:\u003capplication_name\u003e/\u003cdeployment_group_name\u003e\",\n                \"arn:aws:codedeploy:\u003cregion\u003e:\u003caws_account_id\u003e:deploymentconfig:*\",\n                \"arn:aws:codedeploy:\u003cregion\u003e:\u003caws_account_id\u003e:application:\u003capplication_name\u003e\"\n             ]\n          }\n       ]\n    }\n\nFAQ\n---\n\nHow do I add celery?\n\nGo to `compose/production/ecs/django/start` and add the line\n\n`celery multi start worker beat -A config.celery_app`\n\nIf you'd like to troubleshoot your AWS actions, add the\nsecret `ACTION_STEP_DEBUG` with value `true` to your GitHub repo.\n\nHere is the AWS action doc specifying this https://github.com/aws-actions/amazon-ecs-deploy-task-definition#troubleshooting\n\nWhat's this license?\n\nApache 2.0\n\nBest practices?\n\nRotate your keys!\n\nWhat if I mess up creating the ECS service?\n\nGot something there's a service already here? I did too,\nlol. Search up AWS Cloud Map. Delete the one that says `local`.\n\nYou may also have to go to CodeDeploy and delete the Application there, too.\n\nAre you experienced in AWS?\n\nAbsolutely not. This would be my first time actually using AWS besides\nself hosting on one instace. This was just a nice learning experience that seems sooooo\npainful for start ups. In other words, STARTUPS! Get moving! I just gave\nyou a free repo to copy off of :)\n\nI did play around with AWS trying to use the default cookiecutter-django\nbefore which is why I didn't know how I set up ACM in the first place. It\nworked after a painful 12 hours of trying to figure out wtf was going wrong.\n\nWhy do you like typing so much?\n\nI like to train my fingers.\n\nPlus, it's nice seeing my painful moments and learning from them.\nIt's like the cliche standing back and being proud of your work.\n\nBut this was a painful 10 hours... I started at 12 and now it's 22:11.\n\nWhat did you learn from this?\n\nAlways start small. On 10 June 2020, I finally figured to try and start\nsmall with a single EC2 with a load balancer (however, I will admit that\nI suspected the security groups was an issue for the most part).\n\nOn the same day, I finally got it to work. So, always start small, and\nthen try out this methodology.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrew-chen-wang%2Fcookiecutter-django-ecs-github","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fandrew-chen-wang%2Fcookiecutter-django-ecs-github","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fandrew-chen-wang%2Fcookiecutter-django-ecs-github/lists"}