{"id":43352169,"url":"https://github.com/dimagi/ocs-deploy","last_synced_at":"2026-02-02T02:32:37.002Z","repository":{"id":249142328,"uuid":"830082288","full_name":"dimagi/ocs-deploy","owner":"dimagi","description":"AWS CDK Deployment Code for Open Chat Studio","archived":false,"fork":false,"pushed_at":"2025-11-25T07:28:46.000Z","size":223,"stargazers_count":0,"open_issues_count":4,"forks_count":1,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-11-27T22:54:30.279Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dimagi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-07-17T14:52:58.000Z","updated_at":"2025-11-25T07:28:50.000Z","dependencies_parsed_at":"2025-01-16T16:35:16.383Z","dependency_job_id":"e3418fde-2596-45dc-b6c8-d8aadf3ab865","html_url":"https://github.com/dimagi/ocs-deploy","commit_stats":null,"previous_names":["dimagi/ocs-deploy"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dimagi/ocs-deploy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimagi%2Focs-deploy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimagi%2Focs-deploy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimagi%2Focs-deploy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimagi%2Focs-deploy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dimagi","download_url":"https://codeload.github.com/dimagi/ocs-deploy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dimagi%2Focs-deploy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29001654,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-02T01:32:03.847Z","status":"online","status_checked_at":"2026-02-02T02:00:07.448Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-02-02T02:32:36.293Z","updated_at":"2026-02-02T02:32:36.997Z","avatar_url":"https://github.com/dimagi.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Open Chat Studio CDK Deploy\n\nThis project contains the AWS Cloud Development Kit (CDK) stack for deploying the\n[Open Chat Studio](https://github.com/dimagi/open-chat-studio/) infrastructure.\n\n## Table of Contents\n1. [Architecture Overview](#architecture-overview)\n2. [Project Structure](#project-structure)\n3. [Quickstart](#quickstart)\n4. [First Time Deployment Steps](#first-time-deployment-steps)\n5. [Steady State Deployment Steps](#steady-state-deployment-steps)\n6. [Connecting to Running Services](#connecting-to-running-services)\n7. [Managing Multiple Environments](#managing-multiple-environments)\n8. [Adding Environment Variables](#adding-environment-variables)\n9. [Other Useful CDK Commands](#other-useful-cdk-commands)\n10. [Troubleshooting](#troubleshooting)\n\n## Architecture Overview\n\nThe architecture consists of various AWS components:\n\n- **RDS PostgreSQL** database\n- **Elasticache Redis**\n- **Elastic Container Registry (ECR)**\n- **ECS Fargate** services for the Django application, including:\n  - **Django web service**: Runs the Gunicorn server behind an Application Load Balancer (ALB) with a health check target group.\n  - **Celery worker service**\n  - **Celery beat service**\n  - **Migration task**: A one-off task definition for running Django migrations before deployments.\n\nAdditional components set up by this project include:\n\n- VPC with public and private subnets\n- Load balancer for the Django service\n- S3 buckets for media files\n- Certificate Manager certificates for domain management\n- Email identity verification for the domain\n- GitHub Actions roles for the CI/CD pipeline\n- Secrets Manager for storing Django secrets\n\n## Project Structure\n\nThis section describes the layout of the project:\n\n- `ocs_deploy/cli/*.py`: Scripts for deploying CDK stacks and managing secrets.\n  - Run `ocs -l` to see available tasks.\n- `cdk.json`: Configuration file for the CDK Toolkit.\n- `ocs_deploy/`: Contains the CDK stack definitions.\n\n## Quickstart\n\nThis guide assumes you have [uv](https://docs.astral.sh/uv/getting-started/installation/) installed.\n\n### 1. Set Up the Tools\n\n```bash\n$ uv venv\n$ uv pip install -e .\n$ source .venv/bin/activate\n$ ocs -l\n```\n\n### 2. Create Your Configuration\n\n```bash\n$ ocs init \u003cenv\u003e\n```\n\nEdit the generated `.env.{env name}` file to set your required configurations.\n\n## First Time Deployment Steps\n\n### Prerequisites\n\n- You have an AWS account with the necessary permissions and configured SSO.\n- You have the correct AWS profile set. The `--profile X` argument can also be used via the command line, and the default profile is `ocs-{env name}`.\n  ```bash\n  export AWS_PROFILE=XXX\n  ```\n- SSO credentials are configured (`aws configure sso`).\n- You have permissions to create resources in the account.\n- For new environments, run:\n  ```bash\n  ocs --env \u003cenv\u003e aws.bootstrap\n  ```\n\n### Deployment Steps\n\n1. **Set Up RDS, Redis, S3 and the ECR repository**\n\n    ```bash\n    ocs --env \u003cenv\u003e aws.deploy --stacks ec2tmp,rds,redis,s3,ecr\n    ```\n   \n2. Next, push the initial version of the Docker image to the registry:\n\n    ```bash\n    export AWS_ACCOUNT_ID=xxx \\\n      \u0026\u0026 export AWS_REGION=us-east-1 \\\n      \u0026\u0026 export AWS_PROFILE=xxx \\\n      \u0026\u0026 export OCS_ENV=\u003cenv e.g. dev\u003e \\\n      \u0026\u0026 export OCS_NAME=\u003cname e.g. chatbots\u003e \\\n      \u0026\u0026 export REGISTRY=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com \\\n      \u0026\u0026 export IMAGE=$REGISTRY/$OCS_NAME-$OCS_ENV-ecr-repo\n    docker build . -t \"$IMAGE:latest\" -f Dockerfile --build-arg SECRET_KEY=123 --build-arg=DJANGO_ALLOWED_HOSTS=\"dummy\"\n    aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $REGISTRY\n    docker push \"$IMAGE\" --all-tags\n    ```\n\n    For more details on Docker image pushing, visit the [AWS ECR documentation](https://docs.aws.amazon.com/AmazonECR/latest/userguide/docker-push-ecr-image.html).\n\n3. **Set Up Domains and GitHub Roles**\n\n    ```bash\n    ocs --env \u003cenv\u003e aws.deploy --stacks domains,github\n    ```\n   - Create DNS entries for domain and email domain verification.\n   - CNAME records will be provided in the stack output.\n\n4. **Create Necessary Secrets**\n\n    ```bash\n    ocs --env \u003cenv\u003e secrets.create-missing\n    ```\n\n5. **Set Up the Django Service**\n\n    ```bash\n    ocs --env \u003cenv\u003e aws.deploy --stacks django\n    ```\n\n6. **Run Initial Migrations**\n\n    ```bash\n    ocs --env \u003cenv\u003e aws.migrate\n    ```\n\n## Steady State Deployment Steps\n\nAfter the initial deployment, you can deploy any stack independently. Typically, you will only run the CDK deploy when changing infrastructure. For code deployments, use the GitHub Actions defined in the [Open Chat Studio](https://github.com/dimagi/open-chat-studio/) repository.\n\n### Running Migrations\n\nMigrations are run as a one-off ECS task, separate from the web service deployment. This ensures migrations only run once rather than on every container startup.\n\n**For automated deployments**: The GitHub Actions workflow handles running migrations before updating services.\n\n**For manual deployments**: Run migrations before deploying new code:\n\n```bash\nocs --env \u003cenv\u003e aws.migrate\n```\n\nThis command will:\n- Run the migration task in ECS\n- Wait for completion\n- Display the migration logs\n- Report success or failure\n\n## Connecting to Running Services\n\nTo connect to a running service, use the `ocs connect` command:\n\n```bash\nocs --env \u003cenv\u003e connect  # Default command is /bin/bash\nocs --env \u003cenv\u003e connect --command \"python manage.py shell\"\n```\n\n## Managing Multiple Environments\n\nThis project supports multiple deployment environments (e.g., `dev`, `prod`) using separate `.env` files:\n\n- `.env.dev` for the development environment\n- `.env.prod` for the production environment\n\nThe environment name is passed using the `--env` argument, e.g.:\n\n```bash\nocs --env dev aws.deploy\n```\n\nThe environment name is also used to set the default AWS CLI profile:\n\n```bash\nAWS_PROFILE=\"ocs-\u003cenv\u003e\"  # e.g., \"ocs-dev\"\n```\n\n## Adding Environment Variables\n\n### Non-Secret Environment Variable\n\n1. Add the variable to `.env.\u003cenv\u003e` and `.env.example`.\n2. Update the `ocs_deploy.config.OCSConfig` class with the new variable.\n3. Update the `ocs_deploy/fargate.py` file to include the variable in the `env_dict` method.\n\nDeploy the Django service to apply changes:\n\n```bash\nocs --env \u003cenv\u003e aws.deploy --stacks django\n```\n\n### Secret Environment Variable\n\n1. Add the secret's name to the `ocs_deploy/secrets.yml`.\n2. Set the secret value:\n\n    ```bash\n    ocs --env \u003cenv\u003e secrets.set SECRET_NAME SECRET_VALUE\n    ```\n\nAfter setting, deploy the Django service to include the new secret:\n\n```bash\nocs --env \u003cenv\u003e aws.deploy --stack django\n```\n\nIf this step fails ensure that all secrets are set. To re-run this step you will need to manually delete the\n`$NAME-$ENV-CeleryWorkerLogs`, `$NAME-$ENV-DjangoLogs`, and `$NAME-$ENV-CeleryBeatLogs` log groups in CloudWatch.\n\n## Other Useful CDK Commands\n\n- `cdk ls`: List all stacks in the app.\n- `cdk synth`: Emit the synthesized CloudFormation template.\n- `cdk deploy`: Deploy the stack to your default AWS account/region.\n- `cdk diff`: Compare the deployed stack with the current state.\n- `cdk docs`: Open CDK documentation.\n\n## Troubleshooting\n\n**Common Issues**\n\n1. **AWS Credentials Not Set**: Ensure that your AWS credentials are correctly configured and that AWS_PROFILE is set.\n2. **Docker Login Issues**: If you encounter issues with Docker login, ensure that you are using the correct AWS region and account ID.\n3. **Secrets Not Found** : If a secret is not found, ensure that it is correctly defined in the secrets.yml file and that it has been created in AWS Secrets Manager.\n4. **Log group exists**: Log groups are set to RETAIN to prevent accidental deletion but this means that if a stack is rolled back, rolling it out again will fail because the log groups now exist. To fix this, you must manually remove the log groups.\n\n    ```\n    aws logs delete-log-group --log-group-name \u003clog group name\u003e --profile \u003caws profile\u003e\n    ```\n\nFor more detailed troubleshooting, refer to the [AWS CDK documentation](https://docs.aws.amazon.com/cdk/latest/guide/troubleshooting.html).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimagi%2Focs-deploy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdimagi%2Focs-deploy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdimagi%2Focs-deploy/lists"}