{"id":23201589,"url":"https://github.com/mitodl/ocw-studio","last_synced_at":"2025-08-18T23:33:24.168Z","repository":{"id":37465594,"uuid":"285842240","full_name":"mitodl/ocw-studio","owner":"mitodl","description":"Open Source Courseware authoring tool","archived":false,"fork":false,"pushed_at":"2024-10-29T10:54:35.000Z","size":10995,"stargazers_count":9,"open_issues_count":95,"forks_count":3,"subscribers_count":26,"default_branch":"master","last_synced_at":"2024-10-29T12:46:06.487Z","etag":null,"topics":["ocw","opencourseware"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mitodl.png","metadata":{"files":{"readme":"README.md","changelog":"news/__init__.py","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-07T13:59:04.000Z","updated_at":"2024-10-24T21:41:32.000Z","dependencies_parsed_at":"2024-01-29T16:06:43.051Z","dependency_job_id":"d21e86b7-176c-4dec-a3be-b0993a3e628c","html_url":"https://github.com/mitodl/ocw-studio","commit_stats":null,"previous_names":[],"tags_count":308,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitodl%2Focw-studio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitodl%2Focw-studio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitodl%2Focw-studio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mitodl%2Focw-studio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mitodl","download_url":"https://codeload.github.com/mitodl/ocw-studio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230292713,"owners_count":18203650,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ocw","opencourseware"],"created_at":"2024-12-18T15:15:58.254Z","updated_at":"2025-08-18T23:33:24.156Z","avatar_url":"https://github.com/mitodl.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ocw_studio\n\nOCW Studio manages deployments for OCW courses.\n\n**SECTIONS**\n\n- [ocw_studio](#ocw_studio)\n- [Initial Setup](#initial-setup)\n  - [Testing Touchstone login with SAML via SSOCircle](#testing-touchstone-login-with-saml-via-ssocircle)\n  - [Commits](#commits)\n- [Testing and Formatting](#testing-and-formatting)\n  - [JS/CSS Tests and Linting](#jscss-tests-and-linting)\n    - [JS tests](#js-tests)\n    - [JS formatting, linting, and typechecking](#js-formatting-linting-and-typechecking)\n- [Defining local starter projects and site configs](#defining-local-starter-projects-and-site-configs)\n  - [The `localdev` folder](#the-localdev-folder)\n  - [Importing starter projects from Github](#importing-starter-projects-from-github)\n  - [Automatically updating starters from Github](#automatically-updating-starters-from-github)\n- [Enabling GitHub integration](#enabling-github-integration)\n- [Local S3 emulation with Minio](#local-s3-emulation-with-minio)\n- [Enabling Concourse-CI integration](#enabling-concourse-ci-integration)\n  - [Running a Local Concourse Docker Container](#running-a-local-concourse-docker-container)\n  - [End to end testing of site pipelines](#end-to-end-testing-of-site-pipelines)\n  - [Publish And Build Dates](#publish-and-build-dates)\n- [Running OCW Studio on Apple Silicon](#running-ocw-studio-on-apple-silicon)\n- [Video Workflow](#video-workflow)\n- [Enabling YouTube integration](#enabling-youtube-integration)\n- [Enabling Google Drive integration](#enabling-google-drive-integration)\n- [Enabling AWS MediaConvert transcoding](#enabling-aws-mediaconvert-transcoding)\n- [Enabling 3Play integration](#enabling-3play-integration)\n- [Enabling Open Catalog Search Webhooks](#enabling-open-catalog-search-webhooks)\n- [Checking External Resource Availability](#checking-external-resource-availability)\n- [Enabling PostHog Integration](#enabling-posthog-integration)\n\n# Initial Setup\n\n`ocw_studio` follows the same [initial setup steps outlined in the common ODL web app guide](https://mitodl.github.io/handbook/how-to/common-web-app-guide.html).\nRun through those steps **including the addition of `/etc/hosts` aliases and the optional step for running the\n`createsuperuser` command**.\n\nWebsites are created using a template called a \"starter.\" You can import a standard set of starters by running:\n\n```sh\ndocker-compose exec web ./manage.py import_website_starters https://github.com/mitodl/ocw-hugo-projects\n```\n\nThe `ocw-www` starter is meant for creating a home page, aka the \"root website.\" This is called `ocw-www` by default, but the name of the site can be set on `ROOT_WEBSITE_NAME` in your environment if you wish to change it. The other starters are different types of websites that can be built within `ocw-studio`. After you have imported some starters, you are ready to start creating websites. To publish those websites, follow the guides in the table of contents above for setting up:\n\n- A Github organization\n- Google Drive integration for resources\n- AWS S3 credentials (Minio S3 emulation should work out of the box for local development)\n- Youtube / AWS MediaConvert / 3Play if you need to work with videos\n\n### Testing Touchstone login with SAML via SSOCircle\n\n_Note: Testing with ShibTest instead of SSOCircle fails unless python-saml3 is downgraded to 1.2.6 and `use=\"signing\"` is removed from the `KeyDescriptor` tag of the SP metadata_\n\n- NOTE: your app's BASE_URL hostname and the x509 FQDN must match, additionally SSOCircle enforces an requirement that this value be unique per user, so you'll need to pick a hostname no one else on our team is using\n- Create an X.509 certificate \u0026 key with the following command, picking a unique FQDN for yourself (e.g. MYNAME.ocw-studio.odl.local):\n  ```\n  openssl req -new -x509 -days 365 -nodes -out saml.crt -keyout saml.key\n  ```\n- Enter values for the following [SAML configuration variables](http://python-social-auth-docs.readthedocs.io/en/latest/backends/saml.html) in your `.env` file\n  ```sh\n  SOCIAL_AUTH_SAML_SP_ENTITY_ID=http://MYNAME.ocw-studio.odl.local:8043/  # replace with the one entered into the x509 cert above\n  SOCIAL_AUTH_SAML_SP_PUBLIC_CERT=\u003csaml.crt contents, no spaces or returns\u003e\n  SOCIAL_AUTH_SAML_SP_PRIVATE_KEY= \u003csaml.key contents, no spaces or returns\u003e\n  SOCIAL_AUTH_SAML_SECURITY_ENCRYPTED=false\n  SOCIAL_AUTH_SAML_ORG_DISPLAYNAME=ODL Test\n  SOCIAL_AUTH_SAML_CONTACT_NAME=\u003cYour Name\u003e\n  SOCIAL_AUTH_SAML_IDP_ENTITY_ID=https://idp.ssocircle.com\n  SOCIAL_AUTH_SAML_IDP_URL=https://idp.ssocircle.com:443/sso/SSORedirect/metaAlias/publicidp\n  SOCIAL_AUTH_SAML_LOGIN_URL=https://idp.ssocircle.com:443/sso/SSORedirect/metaAlias/publicidp\n  SOCIAL_AUTH_SAML_IDP_ATTRIBUTE_PERM_ID=EmailAddress\n  SOCIAL_AUTH_SAML_IDP_ATTRIBUTE_NAME=FirstName\n  SOCIAL_AUTH_SAML_IDP_ATTRIBUTE_EMAIL=EmailAddress\n  # The value for SOCIAL_AUTH_SAML_IDP_X509 comes from https://idp.ssocircle.com/meta-idp.xml:\n  SOCIAL_AUTH_SAML_IDP_X509=\u003cget value from https://idp.ssocircle.com/meta-idp.xml\u003e\n  ```\n- Go to `http://MYNAME.ocw-studio.odl.local:8043/saml/metadata/` and copy the XML response\n- Register \u0026 login for a free account at `ssocircle.net`, the email that you use to register will be used as your social-auth identifier.\n\n  _SSOCircle free accounts are limited to three concurrent sessions. See https://www.ssocircle.com/en/portfolio/publicidp/idp-pricing/_\n\n- After confirming your registration, go to https://idp.ssocircle.com/sso/hos/ManageSPMetadata.jsp\n  - Click `Add new Service Provider`\n  - Enter your FQDN as the FQDN\n  - Check `FirstName`, `EmailAddress`\n  - Paste the XML response from above into the text field\n  - Submit the form\n- In an incognito browser window, go to `http://MYNAME.ocw-studio.odl.local:8043/login/saml/?next=%2F\u0026idp=default`\n- You should be redirected to SSOCircle, fill out the captcha and click `Continue SAML Single Sign On`\n- You should be redirected back to the /sites/ pages, and be logged in.\n- Log out \u0026 back in as a superuser and to go the Users admin page.\n  - There should be a new user with the same email address and name that you used to register with SSOCircle.\n\n### Commits\n\nTo ensure commits to github are safe, you should install the following first:\n\n```\npip install pre_commit\npre-commit install\n```\n\nTo automatically install precommit hooks when cloning a repo, you can run this:\n\n```\ngit config --global init.templateDir ~/.git-template\npre-commit init-templatedir ~/.git-template\n```\n\n# Testing and Formatting\n\nWriting tests, running the test suite, and formatting code follows the same steps that are outlined in [the common ODL web app guide](https://mitodl.github.io/handbook/how-to/common-web-app-guide.html#testing-and-formatting).\nBelow are some steps that may be particular to this project.\n\n## JS/CSS Tests and Linting\n\nThe JS linting, testing, and formatting tools can be used either in the `watch`\n(node.js) container or on the host computer from the command line.\n\nTo run these things in the Docker container, preface the commands below with\n`docker-compose run --rm watch`.\n\n### JS tests\n\nWe use [Jest](https://jestjs.io/) for our JavaScript tests. It's a nice batteries-included\ntesting framework built for testing React components from the ground up.\n\nTo run the tests:\n\n```sh\nnpm test\n```\n\nFor watch mode (`jest --watch`):\n\n```sh\nnpm run test:watch\n```\n\nTo run a specific test by name:\n\n```sh\nnpm test -- -t \"my test name\"\n```\n\n(note that this will find partial matches too).\n\nTo generate a coverage report:\n\n```sh\nnpm run test:coverage\n```\n\n### JS formatting, linting, and typechecking\n\nWe're using TypeScript for typechecking, eslint for linting, and prettier for\nopinionated code formatting. Just as with the tests above, these commands can\nall be run ether in the Docker container or the host machine.\n\nTo run the typechecker:\n\n```sh\nnpm run typecheck\n```\n\nThis runs `tsc --noEmit`, which basically typechecks the program and outputs\nany error but does not run a full compilation. We have incremental compilation\nturned on, so this should be relatively fast. It uses a file called\n`.tsbuildinfo` for incremental compilation.\n\nTo run the linter:\n\n```sh\nnpm run lint\n```\n\nAnd to format, try:\n\n```sh\nnpm run fmt\n```\n\nYou can also try `npm run fmt:check` to see if any files need to be reformatted.\n\n# Defining local starter projects and site configs\n\nThe `ocw-studio` software allows you to create websites based on a configuration called a \"starter.\" These configuration files are named `ocw-studio.yaml` by default but that name can be overridden by setting `OCW_STUDIO_SITE_CONFIG_FILE` in your environment. These starters can be imported into `ocw-studio` in a couple of different ways.\n\n### The `localdev` folder\n\nMore details on this are in [this readme file](localdev/README.md)\n\n### Importing starter projects from Github\n\nMIT OCW has a set of starter configs that are used in building the official OCW site. They are stored in a repo called [`ocw-hugo-projects`](https://github.com/mitodl/ocw-hugo-projects). This repo can be used as a reference for setting up your own repo. When you make your own repo, make sure that your config files in the repo are in their own folder and the filenames match what is set to `OCW_STUDIO_SITE_CONFIG_FILE`, which is `ocw-studio.yaml` by default. The folder name is used to determine the `slug` property of the resulting starter object, and the config file is read from that folder and applied to the `config` property. When your repo is ready, make sure it is publically accessible and then you can import the starter configs from it by running:\n\n```sh\ndocker-compose exec web ./manage.py import_website_starters https://github.com/mitodl/ocw-hugo-projects\n```\n\nIf you wish to use your own Github repo containing starters use that URL instead. If any starters already exist with the same slug as one being imported, their configuration will be updated.\n\n### Automatically updating starters from Github\n\nIf you are hosting `ocw-studio` on the internet and wish to have your starter updated automatically when you make changes to the starter configurations in your Github repo, this is possible by configuring a webhook. In order to accomplish this, you will need to first set `GITHUB_WEBHOOK_BRANCH` in your environment to the branch that you wish to watch for changes on (i.e. `release`). Then, you will need to configure a webhook in the settings of your Github repo targeting `/api/starters/site_configs` on your instance of `ocw-studio`. After this is set up, on pushes to the configured branch, a webhook will be fired to your `ocw-studio` instance which will trigger automatic updating of your starter configurations.\n\n# Enabling GitHub integration\n\nYou can enable git integration so that website content will be synced with GitHub:\n\n- Create an organization within Github\n- Create a Github personal access token, with all `repo` permissions\n- Add the following to your .env file:\n  ```\n  CONTENT_SYNC_BACKEND=content_sync.backends.github.GithubBackend\n  GIT_ORGANIZATION=\u003cyour_organization\u003e\n  ```\n- If you need to use a custom git domain, add `GIT_API_URL=\u003cyour_domain\u003e/api/v3`\n- If you would like git commits to be anonymized, add `FEATURE_GIT_ANONYMOUS_COMMITS=True`\n\nYou will also need authenticate using either a personal access token or via a github app.\nBoth options have a base rate limit of 5K/hour, but a github app will allow for an additional\n50/hr for each repo in your organization if you have at least 20 repos.\n\nIf you wish to authenticate using a personal access token, create one in Github then set the following\nin your .env file:\n\n```\nGIT_TOKEN=\u003cyour_token\u003e\n```\n\nIf you wish to use a github app,\n[create one for your organization](https://docs.github.com/en/developers/apps/building-github-apps/creating-a-github-app):\n\n- The homepage url can be anything\n- You do not need a callback url or webhook url (disable webhooks)\n- For Repository Permissions, choose \"read/write\" permission for \"Administration\", \"Contents\", \"Pull Requests\", \"Commit Statuses\"\n- After it is created, add the \"App ID\" to your .env file:\n  ```\n  GITHUB_APP_ID=\u003capp id\u003e\n  ```\n- Generate a private key. A pem file will download. You need to use the content of this file in your .env file:\n  ```\n  GITHUB_APP_PRIVATE_KEY=-----BEGIN RSA PRIVATE KEY-----\\nMIIEpQ......\\n-----END RSA PRIVATE KEY-----\n  ```\n- Install the app (follow instructions in link above) to your organization for \"All repositories\"\n\n# Local S3 emulation with Minio\n\nOur `docker-compose` configuration includes an instance of [Minio](https://github.com/minio/minio) which emulates Amazon's S3 service locally.\nThis works in conjunction with the `ENVIRONMENT` env variable being set to \"dev.\" When this is set, usage of the `boto3` library will automatically\nset `endpoint_url` to the internal Docker IP address of the Minio instance. You will need a few env variables:\n\n```python\nMINIO_ROOT_USER=minio_user\nMINIO_ROOT_PASSWORD=minio_password\nAWS_ACCESS_KEY_ID=minio_user\nAWS_SECRET_ACCESS_KEY=minio_password\nAWS_STORAGE_BUCKET_NAME=ol-ocw-studio-app\nAWS_PREVIEW_BUCKET_NAME=ocw-content-draft\nAWS_PUBLISH_BUCKET_NAME=ocw-content-live\nAWS_TEST_BUCKET_NAME=ocw-content-test\nAWS_OFFLINE_PREVIEW_BUCKET_NAME=ocw-content-offline-draft\nAWS_OFFLINE_PUBLISH_BUCKET_NAME=ocw-content-offline-live\nAWS_OFFLINE_TEST_BUCKET_NAME=ocw-content-offline-test\nAWS_ARTIFACTS_BUCKET_NAME=ol-eng-artifacts\nOCW_HUGO_THEMES_BRANCH=main\nOCW_HUGO_PROJECTS_BRANCH=main\nSTATIC_API_BASE_URL=https://ocw.mit.edu\nRESOURCE_BASE_URL_DRAFT=https://draft.ocw.mit.edu\nRESOURCE_BASE_URL_LIVE=https://ocw.mit.edu\n```\n\nNotice how `MINIO_ROOT_USER` is the same value as `AWS_ACCESS_KEY_ID` and `MINIO_ROOT_PASSWORD` is the same as `AWS_SECRET_ACCESS_KEY`. This is to\nensure that the Minio server is initialized with the same access keys that `ocw-studio` is using. The rest of the AWS bucket name keys are the same\nas a standard AWS configuration. the `RESOURCE_BASE_URL` keys are for use with the Concourse container. When using Minio in conjunction with Concourse\nand running any of the management commands that upsert pipelines, these values will be used for the `RESOURCE_BASE_URL` env variable when building sites.\n\nIn sites that support resource upload, you should be able to upload anything except videos to Google Drive using the RC Google Drive credentials, then\nin your site click \"Sync w/ Google Drive.\" If you visit http://localhost:9001 in your web browser, you should be brought to the Minio control panel.\nYou can log into this with whatever you set `MINIO_ROOT_USER` and `MINIO_ROOT_PASSWORD` to. Inside, you should be able to browse the files you uploaded\nto the bucket. Videos are not currently supported locally beacuse of the transcoding service that is normally used with this. The preview and publish\nbuckets are exposed via nginx locally at http://localhost:8044 and http://localhost:8045 respectively.\n\nIn order to complete your local development setup, you will need to follow the instructions below to configure a Concourse Docker container so you\ncan run pipelines and have them push their output to your Minio S3 buckets. The `OCW_HUGO_THEMES_BRANCH` and `OCW_HUGO_PROJECTS_BRANCH` settings will\ncontrol the branch of each of these repos that are pulled down in pipelines that build sites. If you are debugging an issue with a specific branch,\nThis is where you want to change them before you run a command that pushes up a pipeline like `docker-compose exec web ./manage.py backpopulate_pipelines --filter etc...`\n\nNote that you may also want to set `OCW_STUDIO_DRAFT_URL=https://localhost:8044`and `OCW_STUDIO_LIVE_URL=http://localhost:8045` in your `.env` file\nso that the URLs in the publish drawer will point to your Minio published content. If you do this, you will likely need to also set `STATIC_API_BASE_URL_DRAFT=https://draft.ocw.mit.edu`\nand `STATIC_API_BASE_URL_LIVE=https://ocw.mit.edu`. Usually the best way to get started getting content into your local instance of `ocw-studio` is to dump\nand restore the production database to your local instance. One side effect of doing this is that the `ocw-www` site in production has a bunch of different sites linked\nto it via various course lists. When building `ocw-www`, Hugo will attempt to fetch static JSON data related to these linked courses and will encounter errors if it cannot\nfetch them. To avoid this, make sure `STATIC_API_BASE_URL_DRAFT` and `STATIC_API_BASE_URL_LIVE` are set as detailed above. If `STATIC_API_BASE_URL` is not set,\nit will fall back to `OCW_STUDIO_DRAFT_URL` or `OCW_STUDIO_LIVE_URL` depending on the context of the pipeline. So, if you have this set to a URL where the courses\nreferenced in your `ocw-www` site's course lists haven't been published, you will have issues.\n\n# Enabling Concourse-CI integration\n\nConcourse-CI integration is enabled by default to create and trigger publishing pipelines, but you\nwill need to follow some additional steps before it is fully functional.\n\n- Set up Github integration as described above\n- Set up a Concourse-CI instance with a team, username, and password\n- Add the following to your .env file:\n\n  ```\n  AWS_PREVIEW_BUCKET_NAME=\u003cS3 bucket for draft content\u003e\n  AWS_PUBLISH_BUCKET_NAME=\u003cS3 bucket for live content\u003e\n  GIT_DOMAIN=\u003croot domain for github repos, ie github.com\u003e\n  ROOT_WEBSITE_NAME=\u003cWebsite.name for the website that should be the 'home page'\u003e\n\n  CONCOURSE_URL=\u003cThe URL of your Concourse-CI instance\u003e\n  CONCOURSE_TEAM=\u003cConcourse-CI team, defaults to \"ocw\"\u003e\n  CONCOURSE_USERNAME=\u003cConcourse-CI username\u003e\n  CONCOURSE_PASSWORD=\u003cConcourse-CI password\u003e\n  CONCOURSE_IS_PRIVATE_REPO=\u003cTrue if repo is private, False otherwise\u003e\n  API_BEARER_TOKEN=\u003csome hard to guess string\u003e\n  ```\n\n- Draft and live pipelines should then be created for every new `Website` based on a `WebsiteStarter` with `source=github` and a valid github `path`.\n- There are also several management commands for Concourse-CI pipelines:\n  - `backpopulate_pipelines`: to create/update pipelines for all or some existing `Websites` (filters available)\n  - `trigger_pipelines \u003cversion\u003e`: to manually trigger the draft or live pipeline for all or some existing `Websites` (filters available)\n- If you wish to disable concourse integration, set `CONTENT_SYNC_PIPELINE_BACKEND=` in your .env file.\n\n### Running a Local Concourse Docker Container\n\nYou will need to set the following .env variables for the concourse Docker container:\n\n```python\nCONCOURSE_URL=http://concourse:8080\nCONCOURSE_PASSWORD=test\nCONCOURSE_USERNAME=test\nCONCOURSE_TEAM=main\n```\n\nWhen you spin up `ocw-studio` with `docker-compose up`, the Concourse container will come up with everything else.\nThe concourse UI will be available for login at http://concourse:8080 (You should add `127.0.0.1 concourse` to your hosts file.)\nWhen you create a new website or run one of the various management commands that push pipelines up to Concourse, they will go to\nyour local instance instead. The pipeline templates with the `-dev` suffix are used when `settings.ENVIRONMENT` is set to \"dev.\"\n\nWhen you click publish on a site, the pipelines in your local instance of Concourse will be triggered. If you set up Minio as\ndetailed above, the pipelines will publish their output to your locally-running S3 buckets inside it. As also described above,\nyou can view the output of your sites at http://localhost:8044 and http://localhost:8045 for draft and live respectively. You will\nneed to also make sure you run `docker-compose exec web ./manage.py upsert_theme_assets_pipeline` to push up the theme assets\npipeline to your local Concourse instance. You will then need to log into Concourse, unpause the pipeline and start a run of it.\nThis will place theme assets into the bucket you have configured at `AWS_ARTIFACTS_BUCKET_NAME` that your site pipelines can\nreference. If you have already-existing sites that don't have their pipelines pushed up into your local Concourse yet, you will\nneed to run `docker-compose exec web ./manage.py backpopulate_pipelines` and use the `--filter` or `--filter-json` arguments to\nspecify the sites to push up pipelines for. The mass build sites pipeline can be pushed up with `docker-compose exec web ./manage.py upsert_mass_build_pipeline`.\nBeware that when testing the mass build pipeline locally, you will likely need to limit the amount of sites in your local instance\nas using only one dockerized worker publishing the entire OCW site will take a very long time.\n\n### End to end testing of site pipelines\n\nThere is a pipeline definition for end to end testing of sites using the `ocw-www` and `ocw-course` starters. It can be run locally in Concourse using the following steps to set it up.\n\nFirstly, there are some environment variables you will want to set:\n\n```\nOCW_TEST_SITE_SLUGS=[\"ocw-ci-test-www\", \"ocw-ci-test-course\"]\nAWS_TEST_BUCKET_NAME=ocw-content-test\nAWS_OFFLINE_TEST_BUCKET_NAME=ocw-content-offline-test\nSTATIC_API_BASE_URL_TEST=http://10.1.0.102:8046\n```\n\nThere are fixtures for two test websites in the `test_site_fixtures` folder. These contain two sites; `ocw-ci-test-www` and `ocw-ci-test-course` along with test content. In `test_websites.json`, the ID's of the `ocw-www` and `ocw-course` starters are referenced. If these ID's are not correct on your system, you can get the ID's of your starters in Django admin and modify the fixture. They can be loaded into the database with the following commands:\n\n```\ndocker-compose exec web ./manage.py loaddata test_site_fixtures/test_websites.json\ndocker-compose exec web ./manage.py loaddata test_site_fixtures/test_website_content.json\n```\n\nOnce the test sites are in your database, you will need to get them up to your Github org. The easiest way to do this is to run the following commands:\n\n```\ndocker-compose exec web ./manage.py reset_sync_states --filter \"ocw-ci-test-www, ocw-ci-test-course\" --skip_sync\ndocker-compose exec web ./manage.py sync_website_to_backend --filter \"ocw-ci-test-www, ocw-ci-test-course\"\n```\n\nAt this point, you should be able to see the test sites in your Github org and the content should be on the `main` branch. In order to get the content up into the `release` branch, you will need to click the publish button on both sites:\n\nhttp://localhost:8043/sites/ocw-ci-test-www\nhttp://localhost:8043/sites/ocw-ci-test-course\n\nPublishing of the sites will fail because of missing fixtures, but that doesn't matter. All you need to run the end to end testing pipelines is for the content to be in the `release` branch in Github. The last prerequisite you need to set up is to load the static assets into Minio:\n\n- Download the contents of this Google Drive folder: https://drive.google.com/drive/folders/14Hlid31Qy7Yy5V4OgHUwNleYUFuJ5BH2?usp=sharing\n- Browse to the Minio web UI at http://localhost:9001 and log in with your credentials\n- Browse to the `ol-ocw-studio-app` bucket, go to the `courses` folder and create a folder here called `ocw-ci-test-course`\n- In this folder, upload the files you downloaded from Google Drive\n\nYou are now ready to push up the test pipeline to Concourse, which can be done by running:\n\n```\ndocker-compose exec web ./manage.py upsert_e2e_test_pipeline --themes-branch main --projects-branch main\n```\n\nYou can alter the themes branch and projects branches to suit your needs if you are testing a different branch of `ocw-hugo-themes` or `ocw-hugo-projects`. Keep in mind that for any branch of `ocw-hugo-themes` you use, you will need to have built theme assets in Minio. You'll need to run the `upsert_theme_assets` pipeline for that branch and then run it.\n\nYou should now have a pipeline in Concourse called `e2e-test-pipeline`. Run this pipeline and it will:\n\n- Pull down all the necessary git repos\n- Build the test sites\n- Deploy them to Minio\n- Run Playwright tests against the output\n\n### Publish And Build Dates\n\n- Publishing the site from Studio for live and draft versions updates the website properties `(live_build_date, publish_date)` and `(draft_build_date, draft_publish_date)`, respectively.\n- Running the mass build for live and draft versions only updates the website properties `live_build_date` and `draft_build_date`, respectively.\n\n# Running OCW Studio on Apple Silicon\n\nCurrently, the default Docker image for Concourse is not compatible with Apple Silicon. Therefore, run the following command prior to running `docker-compose up`:\n\n```\ncp docker-compose-arm64.yml docker-compose.override.yml\n```\n\n# Video Workflow\n\nThe video workflow for OCW is [described here](/videos/README.md). Note that YouTube integration, Google Drive integration, AWS transcoding, and 3Play integration all need to be set up for the video workflow to work properly. These are described next.\n\n# Enabling YouTube integration\n\n_Note: The steps below describe the process for setting up YouTube integration from scratch. MIT OL Engineers may use YouTube credentials from RC as an acceptable, easier alternative._\n\n- Create a new project at https://console.cloud.google.com/apis/dashboard\n  - Save the project ID in your `.env` file as `YT_PROJECT_ID`\n- Create an OAuth client ID for the project (type: `Web application`)\n  - Add an authorized JavaScript origin (ie `https://\u003cyour_domain\u003e/`)\n  - Add an authorized redirect URI: `https://\u003cyour_domain\u003e/api/youtube-tokens/`\n  - You may need to create an oauth consent screen if prompted; make sure to publish it.\n  - Save your client ID and client secret in your `.env` file (as `YT_CLIENT_ID` and `YT_CLIENT_SECRET`)\n- Enable the YouTube Data API v3 for your project\n- Go to `https://\u003cyour_domain\u003e/api/youtube-tokens/`.\n- You should be prompted to choose a Google account. Choose an account that has upload permissions for your Youtube channel.\n- You will then be prompted to choose an account or brand account. Choose whichever is appropriate.\n- After clicking through these and allowing any requested permissions, you should be redirected back to an API response containing values for YT_ACCESS_TOKEN and YT_REFRESH_TOKEN. Add these to your .env file.\n\n# Enabling Google Drive integration\n\nWith Google Drive integration enabled, a folder on the specified Team Drive will be created for each new website.\nThe folder will have the same name as the `short_id` of the website. Under this folder will be 3 subfolders:\n`files`, `files_final`, `videos_final`. Videos should be uploaded to `videos_final`; everything else should be uploaded\nto `files_final`. The `files` folder is just for temporary storage.\n\nIf this integration is enabled, manual resource creation and file uploads will no longer be possible. Files must\nbe uploaded to Google Drive first, and then the \"Sync w/Google Drive\" button will import and create resources for them.\n\n- Add the following to your .env file:\n\n  ```\n  AWS_STORAGE_BUCKET_NAME=The S3 bucket to upload Google Drive files to. Also populate AWS authentication settings.\n  DRIVE_SHARED_ID=The id of your Google Team Drive\n  DRIVE_SERVICE_ACCOUNT_CREDS=The required Google service account credentials in JSON format.\n  DRIVE_UPLOADS_PARENT_FOLDER_ID=Optional, the folder id in the team drive where course folders should go.\n  ```\n\n- If your site configuration for resources has a non-standard field name for type, add the following to your .env file:\n  ```\n  RESOURCE_TYPE_FIELDS=resourcetype,filetype,\u003cyour_custom_field_name\u003e\n  ```\n\n_Note: MIT OL Engineers may use Google Drive credentials from RC as an alternative to creating their own Google Drive folders._\n\n# Enabling AWS MediaConvert transcoding\n\nThe following environment variables need to be defined in your .env file:\n\n```\nAWS_ACCOUNT_ID\nAWS_ACCESS_KEY_ID\nAWS_SECRET_ACCESS_KEY\nAWS_STORAGE_BUCKET_NAME\nVIDEO_S3_TRANSCODE_ENDPOINT\nAWS_ROLE_NAME\nDRIVE_SHARED_ID\nDRIVE_SERVICE_ACCOUNT_CREDS\nAPI_BEARER_TOKEN\n```\n\nThis will allow for videos to be submitted for transcoding to the AWS MediaConvert service. This is done automatically once a video has been synced to Studio from Google Drive.\n\n# Enabling 3Play integration\n\nThe following environment variables need to be defined in your .env file (for a pre-configured 3Play account):\n\n```\nTHREEPLAY_API_KEY\nTHREEPLAY_CALLBACK_KEY\nTHREEPLAY_PROJECT_ID\n```\n\n# Enabling Open Catalog Search Webhooks\n\nThe following environment variables need to be defined in your .env file in order to notify external course catalogs like MIT Open when OCW sites are created/updated.\n\n```\nOPEN_CATALOG_URLS=delimited list of api endpoint urls that webhooks should be sent to\nOPEN_CATALOG_WEBHOOK_KEY=secret key that will be used to confirm that webhook requests are legitimate\n```\n\n# Checking External Resource Availability\n\nThis feature sets up a cron job to validate external resource urls. The workflow for checking external resource availability is described [here](/external_resources/README.md).\n\n# Enabling PostHog Integration\n\nPostHog is used for dynamically testing and rolling out features that may not be ready for permanent deployment as part of OCW Studio. For example, we use the feature flag `OCW_STUDIO_CONTENT_DELETABLE` to control whether content can be deleted.\n\nThe following delimited list should be set in `.env` file to allow which type of content can be deleted:\n\n```\nOCW_STUDIO_DELETABLE_CONTENT_TYPES=external-resource,instructor,page\n```\n\nThe following variables should be set in the `.env` file for PostHog integration:\n\n```\nPOSTHOG_ENABLED=True\nPOSTHOG_API_HOST=https://app.posthog.com\nPOSTHOG_PROJECT_API_KEY=\u003cobtain from the PostHog dashboard\u003e\n```\n\nThe following variables can be optionally set to configure PostHog requests:\n\n```\nPOSTHOG_FEATURE_FLAG_REQUEST_TIMEOUT_MS=\u003c3000 by default\u003e\nPOSTHOG_MAX_RETRIES=\u003c3 by default\u003e\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitodl%2Focw-studio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmitodl%2Focw-studio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmitodl%2Focw-studio/lists"}