{"id":13478889,"url":"https://github.com/debiki/talkyard-prod-one","last_synced_at":"2025-05-01T12:42:17.967Z","repository":{"id":45013986,"uuid":"57422015","full_name":"debiki/talkyard-prod-one","owner":"debiki","description":"Talkyard production installation on one single server.","archived":false,"fork":false,"pushed_at":"2025-03-20T15:16:31.000Z","size":281,"stargazers_count":262,"open_issues_count":0,"forks_count":43,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-03-27T08:45:32.945Z","etag":null,"topics":["docker-compose","talkyard"],"latest_commit_sha":null,"homepage":"https://www.talkyard.io/","language":"Shell","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/debiki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-MIT.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-04-30T01:28:48.000Z","updated_at":"2025-02-11T15:18:53.000Z","dependencies_parsed_at":"2025-03-16T06:32:34.228Z","dependency_job_id":null,"html_url":"https://github.com/debiki/talkyard-prod-one","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/debiki%2Ftalkyard-prod-one","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/debiki%2Ftalkyard-prod-one/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/debiki%2Ftalkyard-prod-one/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/debiki%2Ftalkyard-prod-one/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/debiki","download_url":"https://codeload.github.com/debiki/talkyard-prod-one/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251878676,"owners_count":21658679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["docker-compose","talkyard"],"created_at":"2024-07-31T16:02:05.283Z","updated_at":"2025-05-01T12:42:17.930Z","avatar_url":"https://github.com/debiki.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"Talkyard production installation\n================\n\nFor one single server: Debian 11 or 12 with at least 2 GB RAM.\n\nDocker based installation. Automatic upgrades.\nAutomatic HTTPS cert (via LetsEncrypt).\nOne installation can host many sites.\n\n\u003c!-- NO, Swarm is abandonware\nIf however you already have a Docker-Compose or Docker Swarm installation\nwith a HTTPS reverse proxy, and want to add Talkyard to it,\nthen have a look at: https://github.com/debiki/talkyard-prod-swarm.\n--\u003e\n\n\nYou should be familiar with Linux, Bash and Git. Otherwise you might run into\nproblems. For example, there might be Git edit conflicts, if you and we change\nthe same file — then you need to know how to resolve those edit conflicts.\nAlso, knowing a bit about Docker can be good.\nSee https://www.talkyard.io/plans for alternatives to installing yourself.\n\nAsk questions and report problems in **[the forum](http://www.talkyard.io/forum/latest/support)**.\nThis is beta software; there might be bugs.\n\n### Security: *Private* server\n\nDon't give people-you-don't-absolutely-trust ssh access to your Talkyard server.\nThe database files in `/opt/talkyard/data/rdb/` are accessible to people who can\nssh into the server, and log files in `/var/log/` are, too.\nThis'll change in Talkyard v1 (next year 2025?) — then we'll use Docker volumes instead.\n\n### Install on your laptop?\n\nHere's [a Vagrantfile here](scripts/Vagrantfile) if you want to test install on a laptop\n— open the Vagrantfile in a text editor, and read, for details.\n(It's old, maybe won't work.)\n\n\n### Install behind an Nginx reverse proxy?\n\n\u003c!-- Someone tried to do this, although in his case, there was *no* reverse proxy. --\u003e\nSkip this, unless you know what a \"reverse proxy\" is;\ninstead, continue below, the section \"Install on a new server\".\nNow, if you _do_ want to install Talkyard on a Debian or Ubuntu server\nwith a Nginx reverse proxy in front of it, with a LetsEncrypt cert — then,\n[here's a mini tutorial](https://www.talkyard.io/-389/talkyard-with-nginx-as-reverse-proxy-and-letsencrypt-for-https-mini-tutorial).\nThe steps 1, 2, 3 ... in that tutorial, are the steps 1, 2, 3 ... below.\n\n\n### Install on a new server\n\nThe rest of this document is about how to install Talkyard on a new server.\n\nInstallation overview: You'll rent a virtual private server (VPS) somewhere, then download\nand install Talkyard, then sign up for a send-emails service and configure email settings.\nThen optionally configure OpenAuth login for Google, Facebook, Twitter, GitHub.\nAnd off-site backups.\n\nDockerfiles, build scripts and source code are in another repo: https://github.com/debiki/talkyard.\nHave a look in `./docker-compose.yml` (in this repo) for details and links.\n\n\nGet a server and a Web address\n----------------\n\nProvision an Debian 11 or 12 server with at least 2 GB RAM, for example at [Digital Ocean](https://www.digitalocean.com/).\n\nPoint a domain name, say, `talkyard.your-website.com`, to the server IP address.\n\n\nInstallation instructions\n----------------\n\n(There's a troubleshooting document here: ./docs/troubleshooting.md )\n\n1. Become root: `sudo -i`, then install Git and English: (can be missing, in minimal Debian builds)\n\n       # As root:\n       apt-get update\n       apt-get upgrade\n       apt-get -y install git vim locales\n       apt-get -y install tree ncdu                # nice to have\n       locale-gen en_US.UTF-8                      # installs English\n       export LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8  # starts using English (warnings are harmless)\n\n1. Create big empty files that you can delete if your server runs out of disk:\n\n       fallocate --length 250MiB /balloon-1-delete-if-disk-full\n       fallocate --length 250MiB /balloon-2-delete-if-disk-full\n       fallocate --length 250MiB /opt/balloon-3-delete-if-disk-full\n       fallocate --length 250MiB /var/balloon-4-delete-if-disk-full\n\n1. Download installation scripts: (you need to install in\n   `/opt/talkyard/` for the backup scripts to work)\n\n       cd /opt/\n       git clone https://github.com/debiki/talkyard-prod-one.git talkyard\n       cd talkyard\n\n1. If you want \u0026 know what you're doing, comment out any swap from `/etc/fstab`, and run: `swapoff -a`.\n\n1. Prepare the OS: install tools, enable automatic security updates, simplify troubleshooting,\n   and make ElasticSearch work: (Consider reading the script first...)\n\n       ./scripts/prepare-os.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n   (...If you don't want to run the whole script, you at least need to copy the\n   sysctl `net.core.somaxconn` and `vm.max_map_count` settings in the script to your\n   `/etc/sysctl.conf` config file — otherwise, the full-text-search-engine (ElasticSearch)\n   won't work. Afterwards, run `sysctl --system` to reload the system configuration.)\n\n1. Install Docker:\n\n       ./scripts/install-docker-compose.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n1. Install a firewall, namely *ufw*: (and answer Yes to the question you'll get. You can skip this if\n   you use Google Cloud Engine; GCE already has a firewall)\n\n   Update 2021-04-04: It's better if you use \u003cb\u003e`firewalld`\u003c/b\u003e instead — it's safer:\n   Docker can bypas `ufw` rules, but not `firewalld` rules.\n   Read more here: https://github.com/chaifeng/ufw-docker,\n   and you can websearch: https://www.google.com/search?q=ufw+docker\n   \u003c!-- [firewalld_not_ufw] [ty_v1] update script, have it use firewalld  --\u003e\n\n       # It's better if you use firewalld instead of this:\n       # (We'll edit the script in a while, so it'll use firewalld instead,\n       # but currently it uses ufw)\n       ./scripts/start-firewall.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n   Here's firewalld: https://firewalld.org/\n\n1. Edit config values:\n\n   ```\n   nano conf/play-framework.conf  # fill in values in the Required Settings section\n   nano .env                      # type a database password\n   ```\n\n   Note:\n   - Set `talkyard.secure=true`, so HTTPS will work — unless you're testing\n     on localhost; then set `talkyard.secure=false`.\n   - If you don't edit `play.http.secret.key` in `play-framework.conf`,\n     the server won't start.\n   - A PostgreSQL database user, named *talkyard*, gets created automatically,\n     by the *rdb* Docker container, with the password you type in the `.env` file.\n     You don't need to do anything.\n   - If you're using a non-standard port, say 8080 (which you do if you're using **Vagrant**),\n     then comment in `talkyard.port=8080` in `play-framework.conf`.\n\n1. Depending on how much RAM your server has (run `free -mh` to find out), choose one of these files:\n   mem/1.7g.yml, mem/2g.yml, mem/3.6g.yml, ... and so on,\n   and copy it to ./docker-compose.override.yml. For example, for\n   a server with 2 GB RAM:\n\n        cp mem/2g.yml docker-compose.override.yml\n\n1. Install and start the latest version. This might take a few minutes\n   the first time (to download Docker images).\n\n        # This script also installs, although named \"upgrade–...\".\n        ./scripts/upgrade-if-needed.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n   (This creates a new Docker network — you can choose the IP range; see the\n   section *A New Docker Network* below.)\n\n   Afterwards, you can type: `docker-compose ps` — you should then see a list\n   of Docker containers in state Up (means they're running).\n\n1. Schedule deletion of old log files, daily backups and deletion old backups,\n   and automatic upgrades:\n\n        ./scripts/schedule-logrotate.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n        ./scripts/schedule-daily-backups.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n        ./scripts/schedule-automatic-upgrades.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n1. Open a web browser; go to `https://talkyard.your website.com` — note: **https**\n   not http.\n\n   Your browser should show a warning about the connection _not_ being secure.\n   Talkyard and LetsEncrypt will now start generating a HTTPS certificate for you.\n   Wait 20 seconds, reload the page, and thereafter HTTPS should work.\n\n   **(** If you'd look in the Nginx log, `tail -f /var/log/nginx/error.log`,\n   you'd see messages like:\n\n   ```\n   domain_whitelist_callback(): Should have cert: talkyard.example.com\n   update_cert_handler(): order rsa cert for talkyard.example.com\n   SSL_do_handshake() failed (SSL: error:... alert bad certificate: SSL alert number 42) while SSL handshaking\n   Replying to ACME HTTP-01 challenge, server name: _, host: talkyard.example.com\n   update_cert_handler(): new rsa cert for talkyard.example.com is saved\n   ```\n   (The \"failed ... alert number 42\" is fine\n   — it's because, at that time, there wasn't yet any cert.) **)**\n\n\n   However, if you're testing on localhost, or with Vagrant,\n   instead go to \u003chttp://localhost\u003e, or \u003chttp://localhost:8080\u003e, respectively.\n   (And you'll need `talkyard.secure=false` in `play-framework.conf`).\n\n1. In the browser, click _Continue_ and create an admin account\n   with the email address you specified when you edited `play-framework.conf` earlier\n   (see above).\n   Follow the getting-started guide.\n\nEverything will restart automatically on server reboot.\n\nNext steps:\n\n\u003c!--\n- Do not enable HTTP2, currently doesn't work with Nginx + the Lua module (apparently [this](https://github.com/openresty/lua-nginx-module/blob/52af63a5b949d6da2289e2de3fb839e2aba4cbfd/src/ngx_http_lua_headers.c#L116) error happens).\n  Update 2021-03: Works fine w OpenResty, if avoiding  ngx.location.capture [63DRN3M75]\n--\u003e\n- Edit `/opt/talkyard/conf/sites-enabled-manual/talkyard-servers.conf` and redirect\n  from HTTP to HTTPS.\u003cbr\u003e\n  (If you for some reason want to run LetsEncrypt's Certbot yourself to generate\n  a HTTPS cert, see [docs/setup-https.md](docs/setup-https.md),\n  and have a look at the commented out `server {}` block at the bottom of\n  `talkyard-servers.conf`.)\n- Sign up for a send-email-service — see the section just below.\n- Send an email to `hello at talkyard.io` so we get your address, and can\n  inform you about security issues and major software\n  upgrades that might require you to do something manually.\n  Or subscribe to the Announcements category over at https://www.talkyard.io/forum/.\n- Copy backups off-site, regularly. See the Backups section below.\n- Configure Gmail, Facebook, Twitter, GitHub login,\n    by creating OpenAuth apps over at their sites, and adding API keys and secrets\n    to `play-framework.conf`. See below, just after the next section, about email.\n- Optionally, create more Talkyard sites hosted by this same Talkyard installation,\n  see [docs/multisite-talkyard.adoc](docs/multisite-talkyard.adoc).\n\n\nConfiguring email\n----------------\n\nIf you don't have a mail server already, then sign up for a transactional email\nservice, for example Mailgun, Elastic Email, SendGrid, Mailjet or Amazon SES.\n(Signing up, and verifying your sender email address and domain, is a bit complicated\n— nothing you do in five minutes.)\n\nThen, configure email settings in `/opt/talkyard/conf/play-framework.conf`,\nthat is, fill in these values:\n\n```\ntalkyard.smtp.host=\"...\"\ntalkyard.smtp.port=\"587\"\ntalkyard.smtp.requireStartTls=true\n#talkyard.smtp.tlsPort=\"465\"\n#talkyard.smtp.connectWithTls=true\ntalkyard.smtp.checkServerIdentity=true\ntalkyard.smtp.user=\"...\"\ntalkyard.smtp.password=\"...\"\ntalkyard.smtp.fromAddress=\"support@your-organization.com\"\n```\n\n(Google Cloud Engine blocks outgoing ports 587 and 465 (at least it did in the past).\nProbably you email provider has made other ports available for you to use,\ne.g. Amazon SES: ports 2587 and 2465.)\n\n\nOpenAuth login\n----------------\n\nYou want login with Facebook, Gmail and maybe Twitter and GitHub to work? Here's how.\n\nHowever, we haven't written easy to follow instructions for this yet.\nSend us an email: `hello at talkyard.io`, mention OpenAuth, and we'll hurry up.\n\n\u003csmall\u003e(There are very very brief instructions in this the markdown source but they might be out of date,\nor there might be typos,\nso they're hidden unless you are a tech person who knows how to view the source.)\u003c/small\u003e\n\n\u003c!-- The \"hidden\" instructons.\nYou can try to follow the instructions below, and maybe won't be easy.\n\nThe login callbacks that you will need to fill in, are\n`http(s)://your.website.com/-/login-auth-callback/NAME` where *NAME* is\none of `google`, `twitter`, `facebook`, `github`.\n\nThe \"copy-paste\" instructions below are for `/opt/talkyard/conf/play-framework.conf`,\nat the end of the file.\n\nFacebook:\n\n - Go to https://developers.facebook.com, and sign up or log in\n - Select the **My Apps** menu to the upper right\n - Click **Add New App**\n - Create a *Products | Facebook Login* app. (We should write more about this and\n   add screenshots.)\n - Copy-paste the Facebook app id into `#facebook.clientID=\"...\"` and `#facebook.clientSecret=\"...\"`\n   (instead of the `...`), and activate (\"comment in\") each line by removing the `#`.\n\nGmail:\n\nFirst, consider visiting https://developers.google.com/people/v1/getting-started#1.-get-a-google-account\n  and reading the instructions.\n\nThen let's get started for real:\n- Go to Google's People API setup tool: https://console.developers.google.com/start/api?id=people.googleapis.com\u0026credential=client_key\n- Select an existing project of yours, or create a new one.\n- Click Continue.\n- You should see a message \"People API has been enabled\" in the upper left corner.\n- Click \"Go to credentials\"\n- You should see: \"Find out what kind of credentials you need\".\n  (If you get lost, you can go back to here, by clicking the upper left corner\n  hamburger menu, then choosing \"APIs \u0026 Services\", then clicking \"Credentials\",\n  then in the \"Create credentials\" dropdown, selecting \"Help me choose\". )\n\n- In the \"Which API are you using?\" dropdown, select \"People API\".\n- In the \"Where will you be calling the API from?\" dropdown, select \"Web server\".\n- Below \"What data will you be accessing?\", select \"User data\".\n- Click \"What credentials do I need\", and proceed with creating credentials if needed.\n\n- Now you need to fill in fields for an OAuth Consent dialog. This dialog is where\n  your users see your organization's name, URL and logo, and can read about\n  how you handle their data — you need to add a link to a Privacy Policy,\n  and Terms of Use. If you don't have your own Privacy Policy and ToU, then,\n  you can use these:\n    https://YOUR_TALKYARD_SERVER/-/privacy-policy\n    https://YOUR_TALKYARD_SERVER/-/terms-of-use\n\n- You'll get to a page \"Client ID for Web application\".\n  There, in the \"Authorized redirect URIs\" field, type:\n    https://YOUR_TALKYARD_SERVER/-/login-auth-callback/google\n\n    (Ignore the \"Authorized JavaScript origins\" field.)\n\n- (Old? blog post w photos:\n    https://medium.com/@pablo127/google-api-authentication-with-oauth-2-on-the-example-of-gmail-a103c897fd98 )\n\nTwitter:\n - Go to https://apps.twitter.com, sign up or log in.\n - Click **Create New App**\n - As callback URL, specify: `https://your.website.com/-/login-auth-callback/twitter`\n - Copy-paste your key and secret into `#twitter.consumerKey=\"...\"` and `#twitter.consumerSecret=\"...\"`,\n   and remove the `#`.\n\nGitHub:\n - Log in to GitHub. Click your avatar menu. Then Settings, then Developer Settings, OAuth Apps.\n - Copy-paste your client ID and secret into `#github.clientID=\"...\"` and `#github.clientSecret=\"...\"`,\n   and remove the `#`.\n--\u003e\n\n\nViewing log files\n----------------\n\nChange directory to `/opt/talkyard/`.\n\nThen, view the application server logs like so: `./view-logs app`\nor `./view-logs -f --tail 30 app`.  \nThe web server: `tail -f /var/log/nginx/{access,error}.log` (mounted on the Docker host in docker-compose.yml)  \nThe database: `less /var/log/postgres/LOG_FILE_NAME`  \nThe search engine: `./view-logs search`.\n\n\nUpgrading to newer versions\n----------------\n\nIf you followed the instructions above — that is, if you ran these scripts:\n`./scripts/prepare-os.sh` and `./scripts/schedule-automatic-upgrades.sh`\n— then your server should keep itself up-to-date, and ought to require no maintenance.\n\nIn a few cases you might have to do something manually, when upgrading.\nLike, running `git pull` and editing config files, maybe running a shell script.\nFor us to be able to tell you about this, please send us an email at\n`hello at talkyard.io`.\n\nIf you didn't run `./scripts/schedule-automatic-upgrades.sh`, you can upgrade\nmanually like so:\n\n    sudo -i\n    cd /opt/talkyard/\n    ./scripts/upgrade-if-needed.sh 2\u003e\u00261 | tee -a talkyard-maint.log\n\n\n\nBackups\n----------------\n\n### Importing a backup\n\nSee [docs/how-restore-backups.md](./docs/how-restore-backup.md).\n\n\nYou can login to Postgres like so:\n\n    sudo docker-compose exec rdb psql postgres postgres  # as user 'postgres'\n    sudo docker-compose exec rdb psql talkyard talkyard  # as user 'talkyard'\n\n\n### Backing up, manually\n\nYou should have configured automatic backups already, see the Installation\nInstructions section above. In any case, you can backup manually like so:\n\n    sudo -i\n    cd /opt/talkyard/\n    ./scripts/backup.sh manual 2\u003e\u00261 | tee -a talkyard-maint.log\n\n\n### Copy backups elsewhere\n\nYou should copy the backups to a safety off-site backup server, regularly.\nOtherwise, if your main server suddenly disappears, or someone breaks into it\nand ransomware-encrypts everything — you'd lose all data.\n\nThere's also a script you can copy-paste to that off-site backup server,\nand run daily via Cron, to get notified via email if backups stop working\n— but no, not yet implmented `[BADBKPEML]`.\n\nSee [docs/copy-backups-elsewhere.md](./docs/copy-backups-elsewhere.md).\n\n\nA new Docker network\n----------------\n\nTalkyard creates its own Docker network, and assigns static IPs to the containers.\nOtherwise, if a container restarts, Docker might give it a new IP,\nand the other containers then couldn't find it it. —\nUnless they're also restarted, so all things that have cached the old stale IP,\npicks up the new IP instead. Or unless one starts using something like Traefik.\nBut static IPs is simpler.\n\nYou can choose the network IP range in the `.env` file — there's this variable:\n\n```\nINTERNAL_NET_SUBNET=172.26.0.0/25\n```\n\n\n\nTips\n----------------\n\nIf you start running out of disk, one reason can be old patches for automatic operating system security updates.\nYou can delete them to free up disk:\n\n```\nsudo apt autoremove --purge\n```\n\n\nDocker mounted directories\n----------------\n\n- `conf/`: Container config files, mounted read-only in the containers. Can add to a Git repo.\n- `data/`: Directories mounted read-write in the containers (and sometimes read-only too). Not for Git.\n\n\n\nLicense (MIT)\n----------------\n\n```\nCopyright (c) 2016-2024 Kaj Magnus Lindberg.\n\nLicensed under the MIT license, see `LICENSE-MIT.txt` — and this is for the\ninstructions and scripts in this repository only, not for Talkyard source code\nor things in other repositories.\n```\n\n\n\u003c!-- vim: set et ts=2 sw=2 tw=0 fo=r list : --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebiki%2Ftalkyard-prod-one","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdebiki%2Ftalkyard-prod-one","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdebiki%2Ftalkyard-prod-one/lists"}