{"id":15632759,"url":"https://github.com/briancaffey/django-step-by-step","last_synced_at":"2025-04-04T10:03:03.920Z","repository":{"id":41448245,"uuid":"352811488","full_name":"briancaffey/django-step-by-step","owner":"briancaffey","description":"A Django + Vue reference project that focuses on developer tooling and CI/CD + IaC","archived":false,"fork":false,"pushed_at":"2025-02-16T07:01:35.000Z","size":23752,"stargazers_count":186,"open_issues_count":2,"forks_count":40,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-28T09:01:49.522Z","etag":null,"topics":["12-factor-app","aws","celery","cicd","django","docker","ecs","github-actions","gunicorn","nginx","python","quasar-framework","terraform","typescript","vue"],"latest_commit_sha":null,"homepage":"https://briancaffey.github.io/django-step-by-step/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/briancaffey.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-03-29T23:27:05.000Z","updated_at":"2025-03-25T15:26:03.000Z","dependencies_parsed_at":"2025-02-16T11:30:44.736Z","dependency_job_id":null,"html_url":"https://github.com/briancaffey/django-step-by-step","commit_stats":null,"previous_names":[],"tags_count":57,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/briancaffey%2Fdjango-step-by-step","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/briancaffey%2Fdjango-step-by-step/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/briancaffey%2Fdjango-step-by-step/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/briancaffey%2Fdjango-step-by-step/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/briancaffey","download_url":"https://codeload.github.com/briancaffey/django-step-by-step/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247149510,"owners_count":20891954,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["12-factor-app","aws","celery","cicd","django","docker","ecs","github-actions","gunicorn","nginx","python","quasar-framework","terraform","typescript","vue"],"created_at":"2024-10-03T10:45:13.133Z","updated_at":"2025-04-04T10:03:03.896Z","avatar_url":"https://github.com/briancaffey.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cspan\u003e\u003cimg src=\"https://img.shields.io/badge/code%20style-black-000000.svg\" /\u003e\n\u003ca href=\"https://github.com/briancaffey/django-step-by-step/actions/workflows/backend_linting_and_unit_tests.yml\"\u003e\u003cimg src=\"https://github.com/briancaffey/django-step-by-step/actions/workflows/backend_linting_and_unit_tests.yml/badge.svg\" /\u003e\u003c/a\u003e\n\u003c/span\u003e\n\nProject Documentation: [https://briancaffey.github.io/django-step-by-step/](https://briancaffey.github.io/django-step-by-step/)\n\n# Django Step by Step\n\nThis is a Django reference project showing how to develop and deploy Django applications using multiple tools and methodologies with best practices and concise examples.\n\n## tl;dr\n\nThis project implements a simple blog app using vanilla Django function-based views and class-based views as well as DRF function and class-based views. There is also an implementation of the same application using GraphQL. The application involves a simple data model including users, posts with text and image and post likes. Local development using virtual environments and docker are both covered in detail, including guidance on how to run the application on different operating systems (Linux, Intel Mac, M1 Mac, Windows, WSL). There is 100% test coverage for the project and comprehensive e2e tests using Cypress.\n\nDeployment to AWS environment on Elastic Container Service (ECS) is shown with both:\n\n- CDK and [a CDK construct library that I wrote specifically for deploying containerized Django projects on AWS](https://github.com/briancaffey/django-cdk)\n- Terraform using [`terraform-aws-django`](https://github.com/briancaffey/terraform-aws-django)\n\nThis project is originally designed as a reference or example project that I can use when I need to recall common patterns, syntax and frequently-used code snippets. I have tried to carefully document each part of the development process as a guide for someone who wants to learn how this project is built and deployed from the ground up. Please visit the project documentation site ([briancaffey.github.io/django-step-by-step/](https://briancaffey.github.io/django-step-by-step/)) for a complete explanation of the project, step-by-step.\n\nThis project is open-source and newcomers to Django and Python and welcome to create issues and pull requests.\n\n## Core application\n\nThis project implements a simple microblogging application called μblog. Here are some of the core features of μblog:\n\n- Anyone (including unauthenticated users) can publish posts with text and an optional image\n- Users can sign up for accounts that are activated by clicking a link in an email\n- Signed-in users can like posts\n- Signed-in users can edit and delete their own posts\n\n## Implementation\n\nμblog's core microblogging application is implemented in 5 different ways:\n\n- Django function-based views\n- Django class-based views\n- Django REST Framework function-based views\n- Django REST Framework class-based views\n- GraphQL with Graphene\n\nEach implementation is tested and documented with OpenAPI (Swagger UI). GraphiQL is exposed for the GraphQL API which includes documentation for queries and mutations.\n\nDjango templates and Bootstrap are used for the function and class based Django views. Vue.js is used to implement the like button using AJAX (with axios).\n\n## Local development\n\nThis project is heavily focused on the process of setting up the development environment. This project has been tested and developed on:\n\n- Ubuntu 20.04\n- macOS 11.4 (Apple Silicon)\n- Windows 10 (WSL + Docker Desktop)\n\nWindows 10 PowerShell development environment is still WIP. Here are some important features of the local development environment:\n\n### Makefile\n\nA Makefile is included in the root of the project to document. This file helps to document each step of the local development environment and deployment process.\n\n### Python dependencies\n\nThis project's Python dependencies are managed with poetry. Poetry dependencies and dev-dependencies are specified in `pyproject.toml`. These dependencies are exported to two files:\n\n- `requirements.txt`: This contains only the dependencies that are required by the project for running in production\n- `requirements_dev.txt`: This contains all the dependencies that are required for running in production and for local development\n\n### Developing with virtual environments\n\nThe application can be developed with a virtual environment locally. This requires starting postgres and redis services locally. Alternatively, postgres, redis and other supporting services can be started with a docker-compose file that exposes the services on `localhost`.\n\n### Developing with docker\n\nDocker is a popular choice for developing and deploying applications, including Django. docker-compose can be used to run the application and dependent services in containers. See `docker-compose.yml` in the root directory for more details.\n\nThe docker-compose file contains the following services:\n\n- postgres: Postgres service\n- redis: Redis service\n- pgadmin: Postgres admin service\n- redis-commander: Redis admin service\n- backend: main Django web application\n- celery_worker: celery worker that processes the default queue\n- celery_beat: celery process that queues tasks on a schedule\n- mailhog: a local SMTP server for testing\n\nThe application can also run locally inside a local Kubernetes cluster. This is made possible with minikube. cdk8s and pulumi are both used to show how kubernetes manifest files can be generated dynamically from code (TypeScript) and deployed to a cluster.\n\n### e2e testing with Cypress\n\nCypress is used for e2e testing which can run locally against the docker-compose development environment. To test user registration and email verification in e2e tests, MailHog and the MailHog API are used to retrieve the email confirmation link from the email sent by the Django application.\n\n## Continuous Integration\n\nMultiple tools are used for running unit tests and code quality checks on each commit. These include:\n\n- GitHub Actions\n- GitLab CI\n- Bitbucket Pipelines\n\nContinuous integration checks that all unit tests pass and that code is formatted correctly. Unit tests run the Python code in a simulated environment that contains the dependent services (postgres and redis). The following tools are used in CI:\n\n- flake8\n- black\n- pytest\n\n## Deployment\n\nThis project focuses on deployment to AWS using Elastic Container Service.\n\n### AWS ECS\n\nECS is my preferred way of running containerized web applications on AWS. To help document best practices for deploying Django applications on ECS, I wrote a reusable CDK construct library that handles infrastructure provisioning for typical deployment scenarios. This project can be found [here](https://github.com/briancaffey/django-cdk).\n\nThe Django ECS construct included in the library creates the following AWS resources:\n\n- VPC (public, private and isolated subnets, NAT Gateways, routing tables, Internet Gateway, etc)\n- S3 bucket for storing static files and media files\n- ECS Cluster (a grouping of ECS resources)\n- ECS tasks and services for various components of the application (web, celery workers, management commands and tasks)\n- A docker image built from our Django application code and specified Dockerfile\n- Application Load Balancer\n- TLS certificate that can attached to the Application Load Balancer terminating SSL/TLS\n- A Route53 DNS record that points to the Application Load Balancer public DNS name\n- AWS RDS Postgres database instance that servers as the database for the Django application\n- A single-node AWS ElastiCache Redis cluster that provides caching and messaging brokering for celery workers\n- Security groups that allow for the stateless application layer to communicate with the Postgres database and Redis cluster\n- Automated database migrations for the Django application using ECS tasks\n\nThe Django ECS construct takes a few inputs, including:\n\n- The path to the Django application code\n- The path to the Dockerfile that defines the main image used for the Django application\n- The domain name in Route53 that will be used to generate DNS records in your account\n- Optionally provide the ARN of an ACM certificate to be attached to the Application Load Balancer\n- A list of strings to be used as the command for the web service\n- A list of strings to be used as the command for the celery workers\n- A list of strings to be used as the command for the celery beat process\n\n\nThis project can deploy ECS from the command line (using `make cdk-deploy`) or from a CI/CD pipeline. GitHub Actions and GitLab CI both include pipeline stages that deploy the application to AWS ECS.\n\nAdditional [ECS Exec](https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/) is used to run management commands and open shells inside of the containers running on Fargate.\n\n## This project is open source and MIT Licensed\n\nSee [LICENSE.md](/LICENSE.md)\n\n## 12 Factor App\n\nThis project tries to follow the [12 Factor App](https://12factor.net/) principles. Here are the 12 factors of the 12 Factor app with a brief description of how they are achieved in this project across multiple deployment environments.\n\n### I: Codebase\n\nThe code can be found in multiple public repositories:\n\n\u003e One codebase tracked in revision control, many deploys\n\n- **GitHub** (please open any issues, pull requests or discussions here): [https://github.com/briancaffey/django-step-by-step](https://github.com/briancaffey/django-step-by-step)\n\n### II: Dependencies\n\nThere are a few different ways in which this project manages dependencies:\n\n- `poetry` manages Python dependencies\n- `npm` manages TypeScript/JavaScript dependencies for the construct library application\n- The `Dockerfile` defines dependencies needed for the container, including the version of Python to use\n- The `django-cdk` library defines specific version of AWS services to use, such as the version of postgres and Redis that are used\n- `terraform-aws-django` also defines specific versions, such as the database engine version to use for Postgres\n\n### III: Config\n\nThere is a good amount of environment variables used for configuration both locally and in the different production environments.\n\nLocally, the application uses default values that allow development in a virtual environment to work without the need for configuring any environment variables as a separate step.\n\nFor example, Postgres is configured with the following default values in the Django application:\n\n```python\nDATABASES = {\n    \"default\": {\n        \"ENGINE\": \"django.db.backends.postgresql_psycopg2\",\n        \"NAME\": os.environ.get(\"POSTGRES_NAME\", \"postgres\"),\n        \"USER\": os.environ.get(\"POSTGRES_USERNAME\", \"postgres\"),\n        \"PASSWORD\": os.environ.get(\"POSTGRES_PASSWORD\", \"postgres\"),\n        \"HOST\": os.environ.get(\"POSTGRES_SERVICE_HOST\", \"localhost\"),\n        \"PORT\": os.environ.get(\"POSTGRES_SERVICE_PORT\", 5432),\n    }\n}\n```\n\nThe `django-cdk` application uses environment variables to determine which options to use for deployment. For example, the `HostedZoneName` is read in from the environment and use to determine which DNS records and ACM records to create. CloudFormation Stacks help to isolate environments, and no infrastructure is shared between environments.\n\n\u003e A litmus test for whether an app has all config correctly factored out of the code is whether the codebase could be made open source at any moment, without compromising any credentials.\n\nThis is another important point from the 12 Factor App principles that is true for this project.\n\nOne interesting part about config is the Postgres password for the ECS environment. The ECS environment uses an AWS Secrets Manager `Secret` to store the database password information, including the password, in a JSON string.\n\nThe name of this `Secret` is passed as an environment to the Django application, and the Django application uses another library to retrieve and cache the database password for later use. Again, the value of the secret is not passed to Django as an environment variable, but the name of the `Secret` (from AWS Secrets Manager) is the value passed. This way we don't even have to worry about what the password is. It is not stored in the code base and it is not even stored anywhere in config (until is created in AWS Secrets Manager).\n\n### IV: Backing Services\n\nIn AWS, it is easy to start relying on AWS services such as RDS, ElastiCache, S3 and other services.\n\n### V: Build, release, run\n\nThis step in the 12 Factor App is where CDK really shines. The `cdk deploy` command builds, releases and runs the application in one command. First, it is important to note that CDK uses something called `DockerImageAsset`, which allows us to point to the directory and Dockerfile of an application that we want to build and push to a docker registry. To use this feature, we must first run `cdk bootstrap` once in our AWS account. This will create the ECR registry that any `DockerImageAsset` will be pushed to.\n\nWhen `cdk deploy` runs, it first builds and pushes the docker image to the ECR registry created by the `cdk bootstrap` command. `cdk deploy` then proceeds to create or update the AWS resources defined in the construct. Another advantage of CDK is automatic rollbacks.\n\n### VI: Processes\n\n\u003e Execute the app as one or more stateless processes\n\nIn each environment, both local and non-local, the application's processes are clearly defined.\n\nThe CDK construct is passed a list of strings that define the processes that are run in the container for the wep services as well as the celery worker and celery beat processes.\n\n### VII: Port binding\n\nThis is done in ECS.\n\n### VIII: Concurrency\n\n\u003e The array of process types and number of processes of each type is known as the process formation.\n\nThe ECS construct has a simple process formation: web service that scales up and down between a minimum and maximum number of instances. Celery workers that scale between a minimum and maximum number of instances, and a single celery beat process.\n\n### IX: Disposability\n\nUsing CDK is a great way to put application disposability into practice. ECS services make sure that some number of ECS Tasks are running at a given point in time. If we go into the AWS console and delete a task, it will be started again almost instantly.\n\n### X: Dev/prod parity\n\nThis project makes dev/prod parity somewhat trivial. Using Infrastructure as Code, we can be confident that two environments will only vary by the values that are passed in through environment variables. If the only difference between two application stacks is the subdomain, then we can expect that everything else about the two applications is similar and that none of the other resources between the two environments will be shared.\n\n### XI: Logs\n\nCloudWatch is a great tool for monitoring logs and getting observability with minimal effort.\n\n### XII: Admin processes\n\n\u003e Run admin/management tasks as one-off processes\n\nDevelopers will often wish to do one-off administrative or maintenance tasks for the app, such as:\n\n- Running database migrations\n- Running a console\n- Running one-time scripts committed into the app’s repo\n\nWith ECS and CDK, management commands can be run easily either through automation or using a tool called ECS Exec.\n\nTo run automated tasks, `managementCommandProps` can be passed a value of `true` (`false` by default). This will run the task each time the application is deployed with `cdk deploy`. This uses the AWS Custom Resource that is defined in the construct. The CR's `onCreate` and `unUpdate` are set to an `AwsSdkCall` that runs `ecs:RunTask` with the appropriate parameters.\n\nIf you do not want to run database migrations automatically as a Custom Resource in CDK, you can use the AWS CLI to run the command from your CI tool, or you can use ECS Exec.\n\nECS Exec would also be a good choice if you want to run a one-time script or open a console in the container.\n\n## Unit Testing backend Django application with GitHub Actions\n\nBackend tests for the Django application run when code is pushed to the `dev` branch.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbriancaffey%2Fdjango-step-by-step","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbriancaffey%2Fdjango-step-by-step","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbriancaffey%2Fdjango-step-by-step/lists"}