{"id":27008785,"url":"https://github.com/chatmail/relay","last_synced_at":"2025-04-13T07:52:02.436Z","repository":{"id":212325958,"uuid":"704034279","full_name":"chatmail/relay","owner":"chatmail","description":"chatmail service deployment scripts and docs ","archived":false,"fork":false,"pushed_at":"2025-04-09T15:41:39.000Z","size":2469,"stargazers_count":201,"open_issues_count":32,"forks_count":21,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-04-09T16:47:31.073Z","etag":null,"topics":["chatmail","dovecot","email","opendkim","postfix","pyinfra"],"latest_commit_sha":null,"homepage":"https://delta.chat/en/2023-12-13-chatmail","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chatmail.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":".github/CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"custom":["https://delta.chat/en/contribute#donate-money"],"liberapay":"delta.chat","open_collective":"delta-chat"}},"created_at":"2023-10-12T12:01:37.000Z","updated_at":"2025-04-09T15:41:42.000Z","dependencies_parsed_at":"2023-12-29T11:29:51.754Z","dependency_job_id":"e2e40bfb-a94b-4ed7-988c-07b0407d1bf7","html_url":"https://github.com/chatmail/relay","commit_stats":{"total_commits":776,"total_committers":10,"mean_commits":77.6,"dds":0.5644329896907216,"last_synced_commit":"7573ef928f0a5d874d500d504fb1f2840db9a4d6"},"previous_names":["deltachat/chatmail","chatmail/chatmail","chatmail/server","chatmail/relay"],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatmail%2Frelay","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatmail%2Frelay/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatmail%2Frelay/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chatmail%2Frelay/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chatmail","download_url":"https://codeload.github.com/chatmail/relay/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248681493,"owners_count":21144700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatmail","dovecot","email","opendkim","postfix","pyinfra"],"created_at":"2025-04-04T09:06:00.192Z","updated_at":"2025-04-13T07:52:02.408Z","avatar_url":"https://github.com/chatmail.png","language":"Python","funding_links":["https://delta.chat/en/contribute#donate-money","https://liberapay.com/delta.chat","https://opencollective.com/delta-chat"],"categories":[],"sub_categories":[],"readme":"\n\u003cimg width=\"800px\" src=\"www/src/collage-top.png\"/\u003e\n\n# Chatmail relays for end-to-end encrypted e-mail\n\nChatmail relay servers are interoperable Mail Transport Agents (MTAs) designed for: \n\n- **Convenience:** Low friction instant onboarding\n\n- **Privacy:** No name, phone numbers, email required or collected \n\n- **End-to-End Encryption enforced**: only OpenPGP messages with metadata minimization allowed \n\n- **Instant:** Privacy-preserving Push Notifications for Apple, Google, and Huawei\n\n- **Speed:** Message delivery in half a second, with optional P2P realtime connections \n\n- **Transport Security:** Strict TLS and DKIM enforced \n\n- **Reliability:** No spam or IP reputation checks; rate-limits are suitable for realtime chats\n\n- **Efficiency:** Messages are only stored for transit and removed automatically\n\nThis repository contains everything needed to setup a ready-to-use chatmail relay\ncomprised of a minimal setup of the battle-tested \n[Postfix SMTP](https://www.postfix.org) and [Dovecot IMAP](https://www.dovecot.org) MTAs/MDAs.\n\nThe automated setup is designed and optimized for providing chatmail addresses\nfor immediate permission-free onboarding through chat apps and bots.\nChatmail addresses are automatically created at first login,\nafter which the initially specified password is required\nfor sending and receiving messages through them.\n\nPlease see [this list of known apps and client projects](https://chatmail.at/clients.html) \nand [this list of known public 3rd party chatmail relay servers](https://chatmail.at/relays).\n\n\n## Minimal requirements, Prerequisites \n\nYou will need the following: \n\n- Control over a domain through a DNS provider of your choice.\n\n- A Debian 12 server with reachable SMTP/SUBMISSIONS/IMAPS/HTTPS ports.\n  IPv6 is encouraged if available.\n  Chatmail relay servers only require 1GB RAM, one CPU, and perhaps 10GB storage for a\n  few thousand active chatmail addresses.\n\n- Key-based SSH authentication to the root user.\n  You must add a passphrase-protected private key to your local ssh-agent\n  because you can't type in your passphrase during deployment.\n  (An ed25519 private key is required due to an [upstream bug in paramiko](https://github.com/paramiko/paramiko/issues/2191))\n\n\n## Getting started \n\nWe use `chat.example.org` as the chatmail domain in the following steps.\nPlease substitute it with your own domain. \n\n1. Setup the initial DNS records.\n   The following is an example in the familiar BIND zone file format with\n   a TTL of 1 hour (3600 seconds).\n   Please substitute your domain and IP addresses.\n\n   ```\n    chat.example.com. 3600 IN A 198.51.100.5\n    chat.example.com. 3600 IN AAAA 2001:db8::5\n    www.chat.example.com. 3600 IN CNAME chat.example.com.\n    mta-sts.chat.example.com. 3600 IN CNAME chat.example.com.\n   ```\n\n2. Clone the repository and bootstrap the Python virtualenv.\n\n   ```\n    git clone https://github.com/chatmail/relay\n    cd relay\n    scripts/initenv.sh\n   ```\n\n3. Create chatmail configuration file `chatmail.ini`:\n\n   ```\n    scripts/cmdeploy init chat.example.org  # \u003c-- use your domain \n   ```\n\n4. Verify that SSH root login works:\n\n   ```\n    ssh root@chat.example.org   # \u003c-- use your domain \n   ```\n\n\n5. Deploy the remote chatmail relay server:\n\n   ```\n    scripts/cmdeploy run\n   ```\n   This script will check that you have all necessary DNS records.\n   If DNS records are missing, it will recommend\n   which you should configure at your DNS provider\n   (it can take some time until they are public).\n\n### Other helpful commands:\n\nTo check the status of your remotely running chatmail service:\n\n```\nscripts/cmdeploy status\n```\n\nTo display and check all recommended DNS records:\n\n```\nscripts/cmdeploy dns\n```\n\nTo test whether your chatmail service is working correctly:\n\n```\nscripts/cmdeploy test\n```\n\nTo measure the performance of your chatmail service:\n\n```\nscripts/cmdeploy bench\n```\n\n## Overview of this repository\n\nThis repository has four directories:\n\n- [cmdeploy](https://github.com/chatmail/relay/tree/main/cmdeploy)\n  is a collection of configuration files\n  and a [pyinfra](https://pyinfra.com)-based deployment script.\n\n- [chatmaild](https://github.com/chatmail/relay/tree/main/chatmaild)\n  is a Python package containing several small services\n  which handle authentication,\n  trigger push notifications on new messages,\n  ensure that outbound mails are encrypted,\n  delete inactive users,\n  and some other minor things.\n  chatmaild can also be installed as a stand-alone Python package.\n\n- [www](https://github.com/chatmail/relay/tree/main/www)\n  contains the html, css, and markdown files\n  which make up a chatmail relay's web page.\n  Edit them before deploying to make your chatmail relay stand out.\n\n- [scripts](https://github.com/chatmail/relay/tree/main/scripts)\n  offers two convenience tools for beginners;\n  `initenv.sh` installs the necessary dependencies to a local virtual environment,\n  and the `scripts/cmdeploy` script enables you\n  to run the `cmdeploy` command line tool in the local virtual environment.\n\n### cmdeploy\n\nThe `cmdeploy/src/cmdeploy/cmdeploy.py` command line tool\nhelps with setting up and managing the chatmail service.\n`cmdeploy init` creates the `chatmail.ini` config file.\n`cmdeploy run` uses a [pyinfra](https://pyinfra.com/)-based [script](`cmdeploy/src/cmdeploy/__init__.py`)\nto automatically install or upgrade all chatmail components on a relay,\naccording to the `chatmail.ini` config.\n\nThe components of chatmail are:\n\n- [Postfix SMTP MTA](https://www.postfix.org) accepts and relays messages\n  (both from your users and from the wider e-mail MTA network)\n\n- [Dovecot IMAP MDA](https://www.dovecot.org) stores messages for your users until they download them\n\n- [Nginx](https://nginx.org/) shows the web page with your privacy policy and additional information\n\n- [acmetool](https://hlandau.github.io/acmetool/) manages TLS certificates for Dovecot, Postfix, and Nginx\n\n- [OpenDKIM](http://www.opendkim.org/) for signing messages with DKIM and rejecting inbound messages without DKIM\n\n- [mtail](https://google.github.io/mtail/) for collecting anonymized metrics in case you have monitoring\n\n- [Iroh relay](https://www.iroh.computer/docs/concepts/relay) \n  which helps client devices to establish Peer-to-Peer connections \n\n- and the chatmaild services, explained in the next section:\n\n### chatmaild\n\n`chatmaild` implements various systemd-controlled services  \nthat integrate with Dovecot and Postfix to achieve instant-onboarding and \nonly relaying OpenPGP end-to-end messages encrypted messages. \nA short overview of `chatmaild` services: \n\n- [`doveauth`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/doveauth.py) \n  implements create-on-login address semantics and is used \n  by Dovecot during IMAP login and by Postfix during SMTP/SUBMISSION login\n  which in turn uses [Dovecot SASL](https://doc.dovecot.org/configuration_manual/authentication/dict/#complete-example-for-authenticating-via-a-unix-socket)\n  to authenticate logins. \n\n- [`filtermail`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/filtermail.py) \n  prevents unencrypted email from leaving or entering the chatmail service\n  and is integrated into Postfix's outbound and inbound mail pipelines.\n\n- [`chatmail-metadata`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metadata.py) is contacted by a\n  [Dovecot lua script](https://github.com/chatmail/relay/blob/main/cmdeploy/src/cmdeploy/dovecot/push_notification.lua)\n  to store user-specific relay-side config.\n  On new messages,\n  it [passes the user's push notification token](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/notifier.py)\n  to [notifications.delta.chat](https://delta.chat/help#instant-delivery)\n  so the push notifications on the user's phone can be triggered\n  by Apple/Google/Huawei.\n\n- [`delete_inactive_users`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/delete_inactive_users.py)\n  deletes users if they have not logged in for a very long time.\n  The timeframe can be configured in `chatmail.ini`.\n\n- [`lastlogin`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/lastlogin.py)\n  is contacted by Dovecot when a user logs in\n  and stores the date of the login.\n\n- [`echobot`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/echo.py)\n  is a small bot for test purposes.\n  It simply echoes back messages from users.\n\n- [`chatmail-metrics`](https://github.com/chatmail/relay/blob/main/chatmaild/src/chatmaild/metrics.py)\n  collects some metrics and displays them at `https://example.org/metrics`.\n\n### Home page and getting started for users\n\n`cmdeploy run` also creates default static web pages and deploys them\nto a Nginx web server with:\n\n- a default `index.html` along with a QR code that users can click to\n  create an address on your chatmail relay \n\n- a default `info.html` that is linked from the home page\n\n- a default `policy.html` that is linked from the home page\n\nAll `.html` files are generated\nby the according markdown `.md` file in the `www/src` directory.\n\n\n### Refining the web pages\n\n```\nscripts/cmdeploy webdev\n```\n\nThis starts a local live development cycle for chatmail web pages:\n\n- uses the `www/src/page-layout.html` file for producing static\n  HTML pages from `www/src/*.md` files\n\n- continously builds the web presence reading files from `www/src` directory\n  and generating HTML files and copying assets to the `www/build` directory.\n\n- Starts a browser window automatically where you can \"refresh\" as needed.\n\n## Mailbox directory layout\n\nFresh chatmail addresses have a mailbox directory that contains: \n\n- a `password` file with the salted password required for authenticating\n  whether a login may use the address to send/receive messages. \n  If you modify the password file manually, you effectively block the user. \n\n- `enforceE2EEincoming` is a default-created file with each address. \n  If present the file indicates that this chatmail address rejects incoming cleartext messages.\n  If absent the address accepts incoming cleartext messages. \n\n- `dovecot*`, `cur`, `new` and `tmp` represent IMAP/mailbox state. \n  If the address is only used by one device, the Maildir directories\n  will typically be empty unless the user of that address hasn't been online \n  for a while. \n\n\n## Emergency Commands to disable automatic address creation\n\nIf you need to stop address creation,\ne.g. because some script is wildly creating addresses, \nlogin with ssh and run:\n\n```\n    touch /etc/chatmail-nocreate\n```\n\nChatmail address creation will be denied while this file is present.\n\n### Ports\n\n[Postfix](http://www.postfix.org/) listens on ports 25 (SMTP) and 587 (SUBMISSION) and 465 (SUBMISSIONS).\n[Dovecot](https://www.dovecot.org/) listens on ports 143 (IMAP) and 993 (IMAPS).\n[Nginx](https://www.nginx.com/) listens on port 8443 (HTTPS-ALT) and 443 (HTTPS).\nPort 443 multiplexes HTTPS, IMAP and SMTP using ALPN to redirect connections to ports 8443, 465 or 993.\n[acmetool](https://hlandau.github.io/acmetool/) listens on port 80 (HTTP).\n\nchatmail-core based apps will, however, discover all ports and configurations\nautomatically by reading the [autoconfig XML file](https://www.ietf.org/archive/id/draft-bucksch-autoconfig-00.html) from the chatmail relay server.\n\n## Email authentication\n\nChatmail relays enforce [DKIM](https://www.rfc-editor.org/rfc/rfc6376)\nto authenticate incoming emails.\nIncoming emails must have a valid DKIM signature with\nSigning Domain Identifier (SDID, `d=` parameter in the DKIM-Signature header)\nequal to the `From:` header domain.\nThis property is checked by OpenDKIM screen policy script\nbefore validating the signatures.\nThis correpsonds to strict [DMARC](https://www.rfc-editor.org/rfc/rfc7489) alignment (`adkim=s`),\nbut chatmail does not rely on DMARC and does not consult the sender policy published in DMARC records.\nOther legacy authentication mechanisms such as [iprev](https://www.rfc-editor.org/rfc/rfc8601#section-2.7.3)\nand [SPF](https://www.rfc-editor.org/rfc/rfc7208) are also not taken into account.\nIf there is no valid DKIM signature on the incoming email,\nthe sender receives a \"5.7.1 No valid DKIM signature found\" error.\n\nOutgoing emails must be sent over authenticated connection\nwith envelope MAIL FROM (return path) corresponding to the login.\nThis is ensured by Postfix which maps login username\nto MAIL FROM with\n[`smtpd_sender_login_maps`](https://www.postfix.org/postconf.5.html#smtpd_sender_login_maps)\nand rejects incorrectly authenticated emails with [`reject_sender_login_mismatch`](reject_sender_login_mismatch) policy.\n`From:` header must correspond to envelope MAIL FROM,\nthis is ensured by `filtermail` proxy.\n\n## TLS requirements\n\nPostfix is configured to require valid TLS\nby setting [`smtp_tls_security_level`](https://www.postfix.org/postconf.5.html#smtp_tls_security_level) to `verify`.\nIf emails don't arrive at your chatmail relay server, \nthe problem is likely that your relay does not have a valid TLS certificate.\n\nYou can test it by resolving `MX` records of your relay domain\nand then connecting to MX relays (e.g `mx.example.org`) with\n`openssl s_client -connect mx.example.org:25 -verify_hostname mx.example.org -verify_return_error -starttls smtp`\nfrom the host that has open port 25 to verify that certificate is valid.\n\nWhen providing a TLS certificate to your chatmail relay server, \nmake sure to provide the full certificate chain\nand not just the last certificate.\n\nIf you are running an Exim server and don't see incoming connections\nfrom a chatmail relay server in the logs,\nmake sure `smtp_no_mail` log item is enabled in the config\nwith `log_selector = +smtp_no_mail`.\nBy default Exim does not log sessions that are closed\nbefore sending the `MAIL` command.\nThis happens if certificate is not recognized as valid by Postfix,\nso you might think that connection is not established\nwhile actually it is a problem with your TLS certificate.\n\n## Migrating a chatmail relay to a new host\n\nIf you want to migrate chatmail relay from an old machine\nto a new machine,\nyou can use these steps.\nThey were tested with a Linux laptop;\nyou might need to adjust some of the steps to your environment.\n\nLet's assume that your `mail_domain` is `mail.example.org`,\nall involved machines run Debian 12,\nyour old site's IP address is `13.37.13.37`,\nand your new site's IP address is `13.12.23.42`.\n\nNote, you should lower the TTLs of your DNS records to a value\nsuch as 300 (5 minutes) so the migration happens as smoothly as possible.\n\nDuring the guide you might get a warning about changed SSH Host keys;\nin this case, just run `ssh-keygen -R \"mail.example.org\"` as recommended.\n\n1. First, disable mail services on the old site. \n\n   ```\n    cmdeploy run --disable-mail --ssh-host 13.37.13.37\n   ```\n\n   Now your users will notice the migration\n   and will not be able to send or receive messages\n   until the migration is completed.\n\n2. Now we want to copy `/home/vmail`, `/var/lib/acme`, `/etc/dkimkeys`, `/run/echobot`, and `/var/spool/postfix` to the new site. \n   Login to the old site while forwarding your SSH agent \n   so you can copy directly from the old to the new site with your SSH key:\n   ```\n    ssh -A root@13.37.13.37\n    tar c - /home/vmail/mail /var/lib/acme /etc/dkimkeys /run/echobot /var/spool/postfix | ssh root@13.12.23.42 \"tar x -C /\"\n   ```\n\n   This transfers all addresses, the TLS certificate, DKIM keys (so DKIM DNS record remains valid), and the echobot's password so it continues to function.\n   It also preserves the Postfix mail spool so any messages pending delivery will still be delivered.\n\n3. Install chatmail on the new machine:\n\n   ```\n    cmdeploy run --disable-mail --ssh-host 13.12.23.42\n   ```\n   Postfix and Dovecot are disabled for now; we will enable them later. \n   We first need to make the new site fully operational. \n\n3. On the new site, run the following to ensure the ownership is correct in case UIDs/GIDs changed:\n\n   ```\n    chown root: -R /var/lib/acme\n    chown opendkim: -R /etc/dkimkeys\n    chown vmail: -R /home/vmail/mail\n    chown echobot: -R /run/echobot\n   ```\n\n4. Now, update DNS entries. \n\n   If other MTAs try to deliver messages to your chatmail domain they may fail intermittently,\n   as DNS catches up with the new site settings \n   but normally will retry delivering messages\n   for at least a week, so messages will not be lost.\n\n5. Finally, you can execute `cmdeploy run --ssh-host 13.12.23.42` to turn on chatmail on the new relay.\n   Your users will be able to use the chatmail relay as soon as the DNS changes have propagated.\n   Voilà!\n\n## Setting up a reverse proxy\n\nA chatmail relay MTA does not track or depend on the client IP address\nfor its operation, so it can be run behind a reverse proxy.\nThis will not even affect incoming mail authentication\nas DKIM only checks the cryptographic signature\nof the message and does not use the IP address as the input.\n\nFor example, you may want to self-host your chatmail relay\nand only use hosted VPS to provide a public IP address\nfor client connections and incoming mail.\nYou can connect chatmail relay to VPS\nusing a tunnel protocol\nsuch as [WireGuard](https://www.wireguard.com/)\nand setup a reverse proxy on a VPS\nto forward connections to the chatmail relay\nover the tunnel.\nYou can also setup multiple reverse proxies\nfor your chatmail relay in different networks\nto ensure your relay is reachable even when\none of the IPs becomes inaccessible due to\nhosting or routing problems.\n\nNote that your chatmail relay still needs\nto be able to make outgoing connections on port 25\nto send messages outside.\n\nTo setup a reverse proxy\n(or rather Destination NAT, DNAT)\nfor your chatmail relay,\nput the following configuration in `/etc/nftables.conf`:\n```\n#!/usr/sbin/nft -f\n\nflush ruleset\n\ndefine wan = eth0\n\n# Which ports to proxy.\n#\n# Note that SSH is not proxied\n# so it is possible to log into the proxy server \n# and not the original one.\ndefine ports = { smtp, http, https, imap, imaps, submission, submissions }\n\n# The host we want to proxy to.\ndefine ipv4_address = AAA.BBB.CCC.DDD\ndefine ipv6_address = [XXX::1]\n\ntable ip nat {\n        chain prerouting {\n                type nat hook prerouting priority dstnat; policy accept;\n                iif $wan tcp dport $ports dnat to $ipv4_address\n        }\n\n        chain postrouting {\n                type nat hook postrouting priority 0;\n\n                oifname $wan masquerade\n        }\n}\n\ntable ip6 nat {\n        chain prerouting {\n                type nat hook prerouting priority dstnat; policy accept;\n                iif $wan tcp dport $ports dnat to $ipv6_address\n        }\n\n        chain postrouting {\n                type nat hook postrouting priority 0;\n\n                oifname $wan masquerade\n        }\n}\n\ntable inet filter {\n        chain input {\n                type filter hook input priority filter; policy drop;\n\n                # Accept ICMP.\n                # It is especially important to accept ICMPv6 ND messages,\n                # otherwise IPv6 connectivity breaks.\n                icmp type { echo-request } accept\n                icmpv6 type { echo-request, nd-neighbor-solicit, nd-router-advert, nd-neighbor-advert } accept\n\n                # Allow incoming SSH connections.\n                tcp dport { ssh } accept\n\n                ct state established accept\n        }\n        chain forward {\n                type filter hook forward priority filter; policy drop;\n\n                ct state established accept\n                ip daddr $ipv4_address counter accept\n                ip6 daddr $ipv6_address counter accept\n        }\n        chain output {\n                type filter hook output priority filter;\n        }\n}\n```\n\nRun `systemctl enable nftables.service`\nto ensure configuration is reloaded when the proxy relay reboots.\n\nUncomment in `/etc/sysctl.conf` the following two lines:\n\n```\nnet.ipv4.ip_forward=1\nnet.ipv6.conf.all.forwarding=1\n```\n\nThen reboot the relay or do `sysctl -p` and `nft -f /etc/nftables.conf`.\n\nOnce proxy relay is set up,\nyou can add its IP address to the DNS.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchatmail%2Frelay","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchatmail%2Frelay","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchatmail%2Frelay/lists"}