{"id":13930173,"url":"https://github.com/desec-io/desec-stack","last_synced_at":"2025-07-19T12:31:50.936Z","repository":{"id":37759409,"uuid":"69393808","full_name":"desec-io/desec-stack","owner":"desec-io","description":"Backbone of the deSEC Free Secure DNS Hosting Service","archived":false,"fork":false,"pushed_at":"2024-11-19T22:07:49.000Z","size":4014,"stargazers_count":396,"open_issues_count":114,"forks_count":50,"subscribers_count":15,"default_branch":"main","last_synced_at":"2024-11-19T23:21:50.513Z","etag":null,"topics":["dns","dnssec","dyndns","rest-api"],"latest_commit_sha":null,"homepage":"https://desec.io/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/desec-io.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["desec-io"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":"deSEC","issuehunt":null,"otechie":null,"custom":["https://desec.io/donate"]}},"created_at":"2016-09-27T20:02:32.000Z","updated_at":"2024-11-19T15:40:11.000Z","dependencies_parsed_at":"2024-03-06T15:29:12.825Z","dependency_job_id":"d41f16f6-5b2c-4afc-86c2-ea5cfba26cd2","html_url":"https://github.com/desec-io/desec-stack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desec-io%2Fdesec-stack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desec-io%2Fdesec-stack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desec-io%2Fdesec-stack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/desec-io%2Fdesec-stack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/desec-io","download_url":"https://codeload.github.com/desec-io/desec-stack/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226607599,"owners_count":17658483,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dns","dnssec","dyndns","rest-api"],"created_at":"2024-08-07T18:05:01.810Z","updated_at":"2025-07-19T12:31:50.918Z","avatar_url":"https://github.com/desec-io.png","language":"Python","funding_links":["https://github.com/sponsors/desec-io","https://liberapay.com/deSEC","https://desec.io/donate"],"categories":["rest-api"],"sub_categories":[],"readme":"deSEC Stack\n===========\n\nThis is a docker compose application providing the basic stack for deSEC name services. It consists of\n\n- `nslord`: Eventually authoritative DNS server (PowerDNS). DNSSEC keying material is generated here.\n- `nsmaster`: Stealth authoritative DNS server (PowerDNS). Receives fully signed AXFR zone transfers from `nslord`. No access to keys.\n- `api`: RESTful API to create deSEC users and domains, see [documentation](https://desec.readthedocs.io/).\n- `dbapi`, `dblord`, `dbmaster`: Postgres databases for `api` and `nsmaster`, MariaDB database for `nslord`, respectively.\n- `www`: nginx instance serving static website content and proxying to `api`\n- `celery`: A shadow instance of the `api` code for performing asynchronous tasks (email delivery).\n- `rabbitmq`: `celery`'s queue\n- `memcached`: `api`-wide in-memory cache, currently used to keep API throttling state\n- `openvpn-server`: OpenVPN server used to tunnel replication traffic between this stack and frontend DNS secondaries\n- `prometheus`: Prometheus server for monitoring\n\nRequirements\n------------\n\nAlthough most configuration is contained in this repository, some external dependencies need to be met before the application can be run. Dependencies are:\n\n1.  We run this software with the `--userland-proxy=false` flag of the `dockerd` daemon, and recommend you do the same.\n\n2.  Also, configure certificates for `openvpn-server`:\n\n    - [Get easy-rsa](https://github.com/OpenVPN/easy-rsa) and follow [this tutorial](https://github.com/OpenVPN/easy-rsa/blob/master/README.quickstart.md).\n    - Then, copy `ca.crt`, `server.crt`, and `server.key` to `openvpn-server/secrets/`.\n    - Create a pre-shared secret using `openvpn --genkey --secret ta.key` inside `openvpn-server/secrets/`.\n\n    For provisioning a secondary, use the same `easy-rsa` PKI and create a new `client.key` and `client.crt` pair. Transfer these securely onto the secondary, along with `ca.crt` and `ta.key`.\n    (You can also create the key on the secondary and only transfer a certificate signing request and the certificate.)\n\n3.  Set sensitive information and network topology using environment variables or an `.env` file. You need (you can use the `.env.default` file as a template):\n    - global\n      - `DESECSTACK_DOMAIN`: domain name under which the entire system will be running. The API will be reachable at https://desec.$DESECSTACK_DOMAIN/api/. For development setup, we recommend using `yourname.dedyn.io`\n      - `DESECSTACK_NS`: the names of the authoritative name servers, i.e. names pointing to your secondary name servers. Minimum 2.\n    - network\n      - `DESECSTACK_IPV4_REAR_PREFIX16`: IPv4 net, size /16, for assignment of internal container IPv4 addresses. **NOTE:** If you change this in an existing setup, you \n        need to manually update persisted data structures such as the MySQL grant tables! Better don't do it.\n      - `DESECSTACK_IPV6_SUBNET`: IPv6 net, ideally /80 (see below)\n      - `DESECSTACK_IPV6_ADDRESS`: IPv6 address of frontend container, ideally 0642:ac10:0080 in within the above subnet (see below)\n      - `DESECSTACK_PORT_XFR`: Port over which XFRs are performed with secondaries\n    - certificates\n      - `DESECSTACK_WWW_CERTS`: `./path/to/certificates` for `www` container. This directory is monitored for changes so that nginx can reload when new keys/certificates are provided. **Note:** The reload is done any time something changes in the directory. The relevant files are **not** watched individually.\n    - API-related\n      - `DESECSTACK_API_ADMIN`: white-space separated list of Django admin email addresses\n      - `DESECSTACK_API_AUTHACTION_VALIDITY`: number of hours for which authenticated action links (e.g. email verification) should be considered valid (default: 0)\n      - `DESECSTACK_API_DEBUG`: Django debug setting. Must be True (default in `docker-compose.dev.yml`) or False (default otherwise)\n      - `DESECSTACK_API_SEPA_CREDITOR_ID`: SEPA creditor ID for donations\n      - `DESECSTACK_API_EMAIL_HOST`: when sending email, use this mail server\n      - `DESECSTACK_API_EMAIL_HOST_USER`: username for sending email\n      - `DESECSTACK_API_EMAIL_HOST_PASSWORD`: password for sending email\n      - `DESECSTACK_API_EMAIL_PORT`: port for sending email\n      - `DESECSTACK_API_SECRETKEY`: Django secret\n      - `DESECSTACK_API_PSL_RESOLVER`: Resolver IP address to use for PSL lookups. If empty, the system's default resolver is used.\n      - `DESECSTACK_DBAPI_PASSWORD_desec`: database password for desecapi\n      - `DESECSTACK_MINIMUM_TTL_DEFAULT`: minimum TTL users can set for RRsets. The setting is per domain, and the default defined here is used on domain creation.\n    - nslord-related\n      - `DESECSTACK_DBLORD_PASSWORD_pdns`: mysql password for pdns on nslord\n      - `DESECSTACK_NSLORD_APIKEY`: pdns API key on nslord\n      - `DESECSTACK_NSLORD_CARBONSERVER`: pdns `carbon-server` setting on nslord (optional)\n      - `DESECSTACK_NSLORD_CARBONOURNAME`: pdns `carbon-ourname` setting on nslord (optional)\n      - `DESECSTACK_NSLORD_DEFAULT_TTL`: TTL to use by default, including for default NS records\n    - nsmaster-related\n      - `DESECSTACK_DBMASTER_PASSWORD_pdns`: mysql password for pdns on nsmaster\n      - `DESECSTACK_NSMASTER_ALSO_NOTIFY`: Comma-separated list of additional IP addresses to notify of zone updates\n      - `DESECSTACK_NSMASTER_APIKEY`: pdns API key on nsmaster (required so that we can execute zone deletions on nsmaster, which replicates to the secondaries)\n      - `DESECSTACK_NSMASTER_CARBONSERVER`: pdns `carbon-server` setting on nsmaster (optional)\n      - `DESECSTACK_NSMASTER_CARBONOURNAME`: pdns `carbon-ourname` setting on nsmaster (optional)\n      - `DESECSTACK_NSMASTER_TSIGKEY`: Base64-encoded value of the default TSIG key used for talking to external secondaries (algorithm: HMAC-SHA256)\n    - monitoring-related\n      - `DESECSTACK_WATCHDOG_SECONDARIES`: space-separated list of secondary hostnames; used to check correct replication of recent DNS changes\n      - `DESECSTACK_PROMETHEUS_PASSWORD`: basic auth password for user `prometheus` at `https://${DESECSTACK_DOMAIN}/prometheus/`\n\nHow to Run\n----------\n\nDevelopment:\n\n    $ ./dev\n\nProduction:\n\n    $ docker compose build \u0026\u0026 docker compose up\n\nStorage\n-------\nAll important data is stored in the databases managed by the `db*` containers. They use Docker volumes which, by default, reside in `/var/lib/docker/volumes/desec-stack_{dbapi_postgres,dblord_mysql,dbmaster_postgres}`.\nThis is the location you will want to back up. (Be sure to follow standard MySQL/Postgres backup practices, i.e. make sure things are consistent.)\n\nAPI Versions and Roadmap\n------------------------\n\ndeSEC currently maintains the following API versions:\n\nAPI Version | URL Prefix | Status    | Support Ends\n----------- | ---------- | --------- | ------------\nVersion 1   | `/api/v1/` |  stable   | earliest 6 months after v2 is declared stable\nVersion 2   | `/api/v2/` |  unstable\n\nYou can find our documentation for all API versions at https://desec.readthedocs.io/. (Select the version of interest in the navigation bar.)\n\nNotes on IPv6\n-------------\n\nThis stack is IPv6-capable. Caveats:\n\n  - It is not necessary to start the Docker daemon with `--ipv6` or `--fixed-cidr-v6`. However, it is recommended to run `dockerd` with `--userland-proxy=false` to avoid \n    exposing ports on the host IPv6 address through `docker-proxy`.\n\n  - Topology: Assuming 2a01:4f8:a0:12eb::/64 is the host network, and we reserve 2a01:4f8:a0:12eb:deec::/80 for the deSEC stack. Docker has more or less established that \n    IPv6 addresses be composed of the /80 prefix and the container MAC address. We choose the private 06:42:ac MAC prefix, defining a /104 subnet. For the remaining 24 \n    bits of the MAC and IPv6 address, the convention seems to be to use the last 24 bits from the internally assigned IPv4 address. However, the first 8 of these are \n    configurable through the `DESECSTACK_IPV4_REAR_PREFIX16` variable. Since we don't want public IPv6 addresses to change if the internal IPv4 net prefix changes, we use \n    `0x10` for bits at position 24--17. We thus arrive at the subnet 2a01:4f8:a0:12eb:deec:642:ac10:0/108 for our public IPv6-enabled Docker containers. The last 16 bits \n    of the IPv6 address we indeed take from the internally assigned IP address. The same procedure is used to set the MAC address of IPv6 containers (they begin with \n    `06:42:ac:10:`).\n\n    All other traffic in the /80 subnet is unexpected and therefore rejected. This includes traffic for IPv6 addresses that Docker assigns. (If Docker uses the MAC address \n    for this purpose, the prefix is 02:42:ac which is not part of our public network, so we're safe.)\n\n    Since the above topology is strictly determined by the /80 prefix and the MAC address, we hope that most of the hardcoding can be removed in the future.\n\n  - Docker currently exposes IPv6-capable containers fully, without restriction. Therefore, it is necessary to set up a firewall, like (`ip6tables`)\n\n        -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT\n        -A FORWARD -d 2a01:4f8:a0:12eb:deec:642:ac10:0/108 -i eth0 -j ACCEPT\n        -A FORWARD -d 2a01:4f8:a0:12eb:deec::/80 -i eth0 -j REJECT --reject-with icmp6-port-unreachable\n\nDevelopment: Getting Started Guide\n----------------------------------\n\nAs desec-stack utilizes a number of different technologies and software packages, it requires some effort to setup a stack ready for development.\nWhile there are certainly many ways to get started hacking desec-stack, here is one way to do it.\n\n1. **Requirements.** This guide is intended and tested on Ubuntu 22.04 LTS.\n    However, many other Linux distributions will also do fine.\n    For desec-stack, [docker and docker compose v2](https://docs.docker.com/engine/install/ubuntu/) are required.\n    Further tools that are required to start hacking are git and curl.\n    Recommended, but not strictly required for desec-stack development is to use certbot along with Let's Encrypt and PyCharm.\n    jq, httpie, libmariadbclient-dev, libpq-dev, python3-dev (\u003e= 3.12) and python3-venv (\u003e= 3.12) are useful if you want to follow this guide.\n    The webapp requires Node.js. To install everything you need for this guide except docker and docker compose, use\n\n       sudo apt install certbot curl git httpie jq libmariadbclient-dev libpq-dev nodejs npm python3-dev python3-venv libmemcached-dev\n\n1. **Get the code.** Clone this repository to your favorite location.\n\n       git clone git@github.com:desec-io/desec-stack.git\n\n1. **Obtain Domain Names.** To run desec-stack, this guide uses a subdomain of dedyn.io provided by desec.io.\n\n    1. Register a deSEC user account. Check out the instructions in our [documentation](https://desec.readthedocs.io/),\n       in particular the [Quickstart](https://desec.readthedocs.io/en/latest/quickstart.html) section.\n\n    2. Register a dedyn.io subdomain to run your desec-stack on it and set up the IP addresses.\n        For this guide, we assume `example.dedyn.io`. Register it with:\n\n           DOMAIN=example.dedyn.io\n           http POST https://desec.io/api/v1/domains/ Authorization:\"Token ${TOKEN}\" name:='\"'${DOMAIN}'\"'\n\n        If you receive an answer that is different from status code 201, chances are that the name you chose is already taken by someone else.\n        In that case, repeat the last step with a new name.\n        To setup the necessary IP address records, we create a couple of A and AAAA records that point to localhost.\n        As preparation, create a JSON file `dns.json` with the following content defining the DNS setup for desec-stack:\n\n           [\n               {\"type\": \"A\",    \"ttl\":300, \"records\": [\"127.0.0.1\"], \"subname\": \"desec\"},\n               {\"type\": \"AAAA\", \"ttl\":300, \"records\": [\"::1\"],       \"subname\": \"desec\"},\n               {\"type\": \"A\",    \"ttl\":300, \"records\": [\"127.0.0.1\"], \"subname\": \"*.desec\"},\n               {\"type\": \"AAAA\", \"ttl\":300, \"records\": [\"::1\"],       \"subname\": \"*.desec\"},\n\n               {\"type\": \"A\",    \"ttl\":300, \"records\": [\"127.0.0.1\"], \"subname\": \"dedyn\"},\n               {\"type\": \"AAAA\", \"ttl\":300, \"records\": [\"::1\"],       \"subname\": \"dedyn\"},\n               {\"type\": \"A\",    \"ttl\":300, \"records\": [\"127.0.0.1\"], \"subname\": \"*.dedyn\"},\n               {\"type\": \"AAAA\", \"ttl\":300, \"records\": [\"::1\"],       \"subname\": \"*.dedyn\"}\n           ]\n\n        We use the deSEC API to publish the DNS information as defined in `dns.json`:\n\n           http POST https://desec.io/api/v1/domains/${DOMAIN}/rrsets/ Authorization:\"Token ${TOKEN}\" \u003c dns.json\n\n1. **Obtain certificates.** desec-stack requires SSL certificates for the above-mentioned `desec` and `dedyn` hostnames as well as for various subdomains.\n    (For a complete list, see `www/README.md`.)\n    While we recommend to obtain signed certificates from Let's Encrypt, it's also possible to let desec-stack generate self-signed certificates on startup\n    by just skipping this step. To use the deSEC certbot hook, first download it to an appropriate location and set up your credentials and domain name.\n\n       mkdir -p ~/bin\n       cd ~/bin\n       curl https://raw.githubusercontent.com/desec-io/desec-certbot-hook/main/hook.sh \u003e desec_certbot_hook.sh\n       touch .dedynauth; chmod 600 .dedynauth\n       echo DEDYN_TOKEN=${TOKEN} \u003e\u003e .dedynauth\n       echo DEDYN_NAME=${DOMAIN} \u003e\u003e .dedynauth\n       chmod +x desec_certbot_hook.sh\n\n    Now we use certbot to obtain certificates, using the DNS challenge for authentication.\n\n       certbot \\\n           --config-dir certbot/config --logs-dir certbot/logs --work-dir certbot/work \\\n           --manual --text --preferred-challenges dns \\\n           --manual-auth-hook ~/bin/desec_certbot_hook.sh \\\n           --manual-cleanup-hook ~/bin/desec_certbot_hook.sh \\\n           --server https://acme-v02.api.letsencrypt.org/directory \\\n           -d \"*.${DOMAIN}\" -d \"update.dedyn.${DOMAIN}\" -d \"update4.dedyn.$DOMAIN\" -d \"update6.dedyn.$DOMAIN\" \\\n           -d \"checkip.dedyn.${DOMAIN}\" -d \"checkipv4.dedyn.${DOMAIN}\" -d \"checkipv6.dedyn.${DOMAIN}\" \\\n           certonly\n\n    Note that the definition of config, logs and work dir are only necessary if you do not want to run certbot as root.\n    Verifying the DNS challenge takes a while, so allow this command to take a couple of minutes.\n    After successfully retrieving the certificate, you can find them in `certbot/config/live/$DOMAIN/`.\n    To make them available to desec-stack (in the default location), we copy certificate and keys.\n    In the project root directory,\n\n       mkdir certs\n       cd certs\n       for SUBNAME in desec www.desec get.desec checkip.dedyn checkipv4.dedyn checkipv6.dedyn dedyn www.dedyn update.dedyn update6.dedyn\n       do\n           ln -s cer ${SUBNAME}.${DOMAIN}.cer\n           ln -s key ${SUBNAME}.${DOMAIN}.key\n       done\n\n       cp ~/bin/certbot/config/live/${DOMAIN}/fullchain.pem cer\n       cp ~/bin/certbot/config/live/${DOMAIN}/privkey.pem key\n\n    The last two steps need to be repeated whenever the certificates are renewed.\n    While any location for the certificates is fine, the `certs/` folder is configured to be ignored by git so that private keys do not accidentally end up being committed.\n\n1. **Configure desec-stack.** As docker compose application, desec-stack is configured by environment variables defined in the `.env` file in the project root directory.\n    Because it contains sensitive information for each deployment, `.env` is not part of the repository and ignored by git.\n    However, we ship `.env.default` and `.env.dev` with templates for production and development, respectively.\n    `.env.dev` is almost good enough for a basic development system, so let's use that as a basis:\n\n       sed \"s/^DESECSTACK_DOMAIN=.*/DESECSTACK_DOMAIN=${DOMAIN}/\" .env.dev \u003e .env\n\n    Optionally, edit the file and\n    1. configure an email server host name, username, and password to deliver emails can be included in `.env`. A convenient option is a MailTrap account.\n    2. adjust the network prefixes in `.env` to avoid collisions with other local networks.\n\n    Additionally, the VPN server for the replication network needs to be equipped with a pre-shared key (PSK) and a public key infrastructure (PKI).\n    To generate the PSK, use the openvpn-server container:\n\n        docker compose build openvpn-server \u0026\u0026 docker compose run openvpn-server openvpn --genkey --secret /dev/stdout \u003e openvpn-server/secrets/ta.key\n\n    To build the PKI, we recommend [easy RSA](https://github.com/OpenVPN/easy-rsa).\n    **Please note that PKI instructions here are for development deployments only!**\n    **Using this setup for production WILL DEFINITELY result in an INSECURE deployment!**\n    To make it available, clone the repository and link to the executable:\n\n        cd openvpn-server/secrets\n        git clone https://github.com/OpenVPN/easy-rsa.git\n        ln -s easy-rsa/easyrsa3/easyrsa\n\n    In order to create a new PKI,\n\n        ./easyrsa init-pki\n        ./easyrsa build-ca nopass\n\n    To make the new PKI's Certificate Authority available to the OpenVPN server,\n\n        ln -s pki/ca.crt\n\n    To issue a certificate for the OpenVPN server, generate a new key pair, a signing request, and sign the certificate.\n\n         ./easyrsa gen-req server nopass\n         ./easyrsa sign-req server server  # requires interaction\n\n    Make the key and certificate available to OpenVPN server:\n\n        ln -s pki/issued/server.crt\n        ln -s pki/private/server.key\n\n    As the setup of OpenVPN is completed, return to the project directory:\n\n        cd -\n\n1. **Install webapp dependencies.** To install the dependencies for the web site and GUI, run\n\n       cd webapp/\n       npm install\n       cd -\n\n1. **Run desec-stack.** To run desec-stack, use\n\n       ./dev\n\n    If you run desec-stack for the first time, this will require a couple of downloads and take a while.\n    Once it is up and running, you can query the API home endpoint:\n\n       http GET https://desec.${DOMAIN}/api/v1/\n\n    Congratulations, you have desec-stack up and running.\n\n    A convenient way to create a test user account is via\n\n       docker compose exec api python3 manage.py shell -c 'from desecapi.models import User; User.objects.create_user(email=\"test@example.com\", password=\"test1234\", limit_domains=None);'\n\n    but users can also be created by signing up via the web GUI.\n    The latter, however, requires that you can read email that is sent from your local setup.\n    This can be achieved, e.g., by using mailtrap.io.\n\n    desec-stack marks `dedyn.$DESECSTACK_DOMAIN` as a locally registerable public suffix.\n    To facilitate the registration process, `$DESECSTACK_DOMAIN` needs to be created via the API.\n    A convenient way to do that using the user created above is\n\n      (source .env \u0026\u0026 docker compose exec api python3 manage.py shell -c \"from desecapi.models import User, Domain; from desecapi.pdns_change_tracker import PDNSChangeTracker; PDNSChangeTracker.track(lambda: Domain.objects.create(name='dedyn.$DESECSTACK_DOMAIN', owner=User.objects.get(email='test@example.com')));\")\n\n    Of course, as this setup is only on your local machine, DNS information will not be published into the public DNS.\n    However, the desec-stack nameserver is available on localhost port 5321.\n    To check if desec-stack is working as expected, you can query the desec-stack nameserver locally for any information that you saved using your API.\n\n       EMAIL=john@example.com\n       PASSWORD=insecure\n       # Register account (https://desec.readthedocs.io/en/latest/quickstart.html). Hint: In dev mode, the captcha response contains the plaintext challenge.\n       TOKEN=$(http POST https://desec.${DOMAIN}/api/v1/auth/login/ email:=\\\"${EMAIL}\\\" password:=\\\"${PASSWORD}\\\" | jq -r .token)\n       http POST https://desec.${DOMAIN}/api/v1/domains/ Authorization:\"Token ${TOKEN}\" name:='\"test.example\"'\n       http POST https://desec.${DOMAIN}/api/v1/domains/test.example/rrsets/ Authorization:\"Token ${TOKEN}\" type:=\\\"A\\\" ttl:=60 records:='[\"127.0.0.254\"]'\n\n    After registering a user with your API, creating a domain and publishing some info to the DNS, use\n\n       dig @localhost -p 5321 test.example \n\n    to see if the nameserver is behaving as expected.\n\n1. **(Optional) Configure PyCharm for API Development.** As a docker compose application, desec-stack takes a while to start.\n    Additionally, it is hard to connect a debugger to the docker containers.\n    Our recommended solution is to develop the API using Django tests running outside the docker compose application.\n    This will dramatically decrease the time required for running the Django tests and enable just-in-time debugging in PyCharm.\n    Also, it will enable you to browse dependencies and code within PyCharm and thus ease debugging.\n\n    1. To get started, we create a virtual python environment that (to some extent) mimics the python environment in the docker container.\n        In the project root,\n\n           cd api\n           python3 -m venv venv  # Python \u003e= 3.12\n           source venv/bin/activate\n           pip install wheel\n           pip install -r requirements.txt\n\n    1. At this point, Django is ready to run in the virtual environment created above.\n        There are two things to consider when running Django outside the container.\n        First, the environment variables as defined in the `.env` file need to be made available in the shell.\n        This can be done with\n\n           set -a \u0026\u0026 source ../.env \u0026\u0026 set +a\n\n        Second, to make the tests run efficiently, a couple of settings are different from the production system:\n        passwords are hashed using a fast (but insecure!) method, rate limits are switched off, and so on.\n        To use the fast settings in your shell, run\n\n           export DJANGO_SETTINGS_MODULE=api.settings_quick_test\n\n        Third, the API needs a postgres database to run the tests. To serve as a test database,\n        the `dbapi` container can be started using a test configuration which exposes the database at\n        `127.0.0.1`. In order to let Django know that the database is at `127.0.0.1` instead of the\n        usual `dbapi`, set an additional environment variable:\n\n           export DESECSTACK_DJANGO_TEST=1\n\n        Fourth, run the database:\n\n           docker compose -f docker-compose.yml -f docker-compose.test-api.yml up -d dbapi\n\n        Finally, you can manage Django using the `manage.py` CLI.\n        As an example, to run the tests, use\n\n           python3 manage.py test\n\n    1. Open the project root directory `desec-stack` in PyCharm and select File › Settings.\n        1. In Project: desec-stack › Project Structure, mark the `api/` folder as a source folder.\n        2. In Project: desec-stack › Project Interpreter, add a new interpreter. Choose \"existing environment\" and select `api/venv/bin/python3` from the project root.\n        3. In Languages \u0026 Frameworks › Django, enable the Django support and set the Django project root to `api/`.\n\n    1. From the PyCharm menu, select Run › Edit Configurations and click on \"Edit configuration templates\"; select the \"Django tests\" template from the list.\n        1. Open the Environment Variables dialog. Copy the contents of the `.env` file and paste it here.\n        2. Add an environment variable with the name `DESECSTACK_DJANGO_TEST` and the value `1`.\n        3. Fill the Custom Settings field with the path to the `settings_quick_test` module.\n        4. At the bottom in the \"Before launch\" sections, add an \"External tool\" with the following settings:\n           - Name: `Postgres Test Container`\n           - Program: `docker`\n           - Arguments: `compose -f docker-compose.yml -f docker-compose.test-api.yml up -d dbapi`\n\n    1. To see if the test configuration is working, right-click on the api folder in the project view and select Run Test.\n       (Note that the first attempt may fail in case the `dbapi` container does not start up fast enough. In that case, just try again.)\n\n    1. To use code inspection, click on Inspect Code… in PyCharm's Code menu and add a local custom scope with the following pattern:\n\n           file:api//*.py\u0026\u0026!file:api/venv//*\u0026\u0026!file:api/manage.py\u0026\u0026!file:api/api/wsgi.py\u0026\u0026!file:api/desecapi/migrations//*\n\n    From this point on, you are set up to use most of PyCharm's convenience features.\n\n    1. For PyCharm's Python Console, the environment variables of your `.env` file and `DJANGO_SETTINGS_MODULE=api.settings_quick_test` need to be configured in Settings › Build, Execution, Deployment › Console › Django Console. (Note that if you need to work with the database, you need to initialize it first by running all migrations; otherwise, the model tables will be missing from the database.)\n\n1. **Code quality.** We use [Black](https://pypi.org/project/black/) to ensure formatting consistency and minimal diffs. Before you commit Python code into the `api/` directory, please run `black api/desecapi/`.\n\n\n## Debugging\n\n### RabbitMQ\n\nTo access message queue information of RabbitMQ, use the RabbitMQ management plugin. First, port 15672 of the RabbitMQ\ncontainer needs to be exposed (default when using `docker-compose.dev.yml`). Then, inside the container, create a user\nthat can access the RabbitMQ data:\n\n```\nrabbitmq-plugins enable rabbitmq_management\nrabbitmqctl add_user admin admin\nrabbitmqctl set_user_tags admin administrator\nrabbitmqctl set_permissions admin '.*' '.*' '.*'\n```\n\nThen the web-based management interface will be available at http://localhost:15672 with user `admin` and password\n`admin`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdesec-io%2Fdesec-stack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdesec-io%2Fdesec-stack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdesec-io%2Fdesec-stack/lists"}