{"id":15136262,"url":"https://github.com/mipro98/infra","last_synced_at":"2025-10-07T05:43:25.189Z","repository":{"id":182833941,"uuid":"665989374","full_name":"mipro98/infra","owner":"mipro98","description":"My IaC configuration for my homelab.","archived":false,"fork":false,"pushed_at":"2024-10-18T20:18:41.000Z","size":266,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-30T18:05:15.467Z","etag":null,"topics":["ansible","ansible-vault","btrbk","btrfs","homelab-automation","nextcloud"],"latest_commit_sha":null,"homepage":"","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mipro98.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-07-13T12:59:25.000Z","updated_at":"2024-10-18T20:18:44.000Z","dependencies_parsed_at":"2024-01-03T16:47:00.590Z","dependency_job_id":"c08c6cfc-13ab-48f2-9ebd-62069f19c89a","html_url":"https://github.com/mipro98/infra","commit_stats":null,"previous_names":["mipro98/infra"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mipro98%2Finfra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mipro98%2Finfra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mipro98%2Finfra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mipro98%2Finfra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mipro98","download_url":"https://codeload.github.com/mipro98/infra/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237821496,"owners_count":19371772,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","ansible-vault","btrbk","btrfs","homelab-automation","nextcloud"],"created_at":"2024-09-26T06:20:16.366Z","updated_at":"2025-10-07T05:43:25.180Z","avatar_url":"https://github.com/mipro98.png","language":"Jinja","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=center\u003e🖴 mipro98/infra 🖴\u003c/h1\u003e\n\u003ch3 align=center\u003eAn Ansible playbook for a docker-based homelab with automatic maintenance and monitoring.\u003c/h3\u003e\n\n---\n\n\u003ch4 align=center\u003eFlexible docker-compose templates for various services like Nextcloud, Vaultwarden, Gitea, Prometheus+Grafana,...\u003c/h4\u003e\n\u003ch4 align=center\u003eAll services are set up ready-to-use with a Traefik reverse proxy and automatic TLS certificates.\u003c/h4\u003e\n\u003ch4 align=center\u003eUnattended server maintenance with auto updates, S.M.A.R.T monitoring, BTRFS snapshots and email notifications.\u003c/h4\u003e\n\n---\n\n## Features\n\nThis playbook includes tasks for:\n* Setting up the host system (packages, user, mounts,...).\n* Deploying all docker services according to their docker-compose templates.\n* Setting up fully automatic maintenance tasks like:\n  * BTRFS snapshots and backups to a different drive using the [btrbk utility](https://digint.ch/btrbk/)\n  * BTRFS scrubbing\n  * S.M.A.R.T. monitoring with email notifications\n  * automatic system and container updates.\n\n\nVarious configuration can be done using variables defined in `group_vars` or in `vault.yml`. For almost all files, [templates](https://docs.ansible.com/ansible/latest/collections/ansible/builtin/template_module.html) are used to craft configuration files or docker-compose.yml files according to set variables.\n\n\n\u003cu\u003e**This repo contains ready-to-use container setups for:**\u003c/u\u003e\n* Nextcloud ([with Mariadb, Redis, Cron, Collabora, Memories incl. hardware transcoding](#nextcloud-stack))\n* A server monitoring stack with [Prometheus](https://prometheus.io/), [Grafana](https://grafana.com/) and [node_exporter](https://github.com/prometheus/node_exporter).\n* [Dokuwiki](https://www.dokuwiki.org/dokuwiki)\n* [Gitea](https://about.gitea.com/)\n* [Homer](https://github.com/bastienwirtz/homer)\n* [Vaultwarden](https://github.com/dani-garcia/vaultwarden)\n* [ulogger](https://github.com/bfabiszewski/ulogger-server)\n* [OwnTracks](https://owntracks.org/) ([recorder](https://github.com/owntracks/recorder) + [frontend](https://github.com/owntracks/frontend))\n* [Planka](https://planka.app/)\n* [Firefly III](https://www.firefly-iii.org/)\n* [Traefik v2](https://doc.traefik.io/traefik/) as the central reverse proxy.\n\n**_All these services are setup to be exposed through Traefik on different domains by simply providing domain names for the services in `vault.yml`._**\n\n## Quick start\n\n1. Clone the repo.\n2. Run `./git-init.sh`\n3. (optionally: run `source unlock-bw.sh`)\n4. Deploy (see below)\n\n\nDeploy everything:\n```bash\nmake deploy\n```\n\n* Deploy only host os system setup: `make system`\n* Deploy only docker-compose changes: `make containers`\n* Deploy only maintenance changes: `make maintenance`\n\n\n## Ansible-Vault\n\nThis repo uses ansible-vault to encrypt secret variables used by Ansible (passwords, domain names, etc.). This affects the file `vars/vault.yml`. A template for `vault.yml` can be found in `vars/vault.yml.template` The password for the encryption is obtained by [bitwarden-cli](https://github.com/bitwarden/cli) in the script `vault-pass.sh`. Therefore you can encrypt and decrypt by just entering your bitwarden master password by just writing:\n\n```bash\nmake encrypt\nmake decrpyt\n```\n\n**Always remember to encrypt the vault before commiting! Also, absolutely run the script `git-init.sh` after cloning to install a pre-commit hook which prevents committing the unencrypted vault!**\n\n---\n## Roles\n\nThe repo contains 3 ansible roles:\n1. `system`\n2. `containers`\n3. `maintenance`\n\n\n### `system` role\n\n  * installs all packages defined in `extra_packages`\n  * sets correct permissions for docker\n  * sets fstab mount points for a \"NAS\" and a \"BACKUP\" drive\n  * sets up `smartd` with a sane monitoring configuration and email alerts\n\n_Note:_ SSH configuration is skipped for now.\n\n\n### `container` role\n\n* Creates the folder structure for docker-compose files according to enabled services\n* Deploys the docker-compose.yml templates for the services in their respective folders\n* Makes some special configuration for nextcloud, if nextcloud is enabled\n\n\n### `maintenance` role\n\nAt the heart of the maintenance role is the script `mpserver-maintenance.sh` which gets deployed by ansible. Additionally:\n* A systemd timer is set up which triggers above script every night at 2 A.M.\n* The `btrbk.conf` file is deployed\n* An `uptime-monitor.sh` script is deployed which checks continously whether the server is online. When it is offline, it optionally executes an ssh command  (I use it to restart my router). After successful reconnect, an E-Mail is sent about the downtime.\n* A systemd service is installed to start the `uptime-monitor.sh` script on boot.\n\nRefer to section [Automatic Maintenance](#automatic-maintenance) for more details.\n\n---\n## Automatic Maintenance\n\nThe custom bash script `mpserver-maintenance.sh` gets triggered every night at 2 A.M. by systemd. Based on the time (or command line flags), it decides what actions to take:\n\n* monthly tasks _(when script is triggered on last sunday of a month)_\n* weekly tasks _(when script is triggered on a sunday except the last sunday of a month)_\n* daily tasks _(when triggered all other days between 0 A.M. and 6 A.M.)_\n\nThe script can be triggered manually for daily, monthly or weekly tasks. Refer to `sudo ./mpserver-maintenance.sh --help`.\n\n```\nUsage: ./mpserver-maintenance.sh [\u003coptions\u003e] [\u003ccommand\u003e]\n\nOptions:\n\t--dry-run\tDon't execute anything, only show what would be executed\n\t--no-email\tDon't send an email with the log\n\t--help\t\tShow this help message and exit.\n\nCommands:\n\tdaily\t\tRun daily maintenance tasks\n\tweekly\t\tRun weekly maintenance tasks\n\tmonthly\tRun monthly maintenance tasks\n\n\tIf no command is passed, the script will determine the correct schedule and run the appropriate tasks.\n```\n\nThe tasks are defined in the bash functions `weekly`, `monthly`, `daily`. Each trigger writes a detailed log into `~/maintenance/logs` which also gets emailed for `monthly` and `weekly` triggers.\n\n\n**Daily tasks**:\n1. Snapshot the subvolume \"Daten\" through btrbk\n\n**Weekly tasks:**\n1. backup `~/dockerdata` onto a backup location through rsync\n2. snapshot _and_ backup all configured subvolumes according to `btrbk.conf`\n3. update the host system\n4. send an email with a the detailed log.\n\n**Monthly tasks:**\n1. backup `~/dockerdata` onto a backup location through rsync\n2. snapshot _and_ backup all configured subvolumes according to `btrbk.conf`\n3. update all containers (`docker-compose pull` all services)\n4. prune old docker images and volumes\n5. `btrfs-scrub` both the _NAS_ and the _BACKUP_ drive.\n6. update the host system.\n7. send an email with a the detailed log.\n\n---\n\n## Nextcloud stack\n\n\nThe Nextcloud stack is carefully fine-tuned for performance and simplicity using:\n\n* A custom built Nextcloud image with `ffmpeg`, `imagemagick` and `ghostscript` for media previews.\n* **MariaDB** for a performant database\n* **Redis** for fast memory caching and Transactional File Locking _(I personally use APCu for caching and Redis only for File Locking)_\n* A dedicated **go-vod** container with access to `/dev/dri` for hardware acceleration in Nextcloud Memories.\n* A dedicated Nextcloud **cron** container \u003cu\u003ewith mounted crontab file\u003c/u\u003e so you can easily add cronjobs without messing on the host system's cron/systemd.\n* A dedicated **Collabora** container for performant **Nextcloud Office** integration.\n\n**Notes:**\n\n* The nextcloud container will _not_ auto-update unless you run `docker-compose build --pull nextcloud` inside `~/docker/nextcloud` because the container is built using `~/docker/nextcloud/Dockerfile` in order to have ffmpeg support. Since Nextcloud updates often require manual intervention and can easily be discovered through the admin panel, this isn't that much of an issue.\n* I use [Nextcloud Memories](https://memories.gallery/) alongside [Preview Generator](https://apps.nextcloud.com/apps/previewgenerator) and [Recognize](https://apps.nextcloud.com/apps/recognize) without issues and with hardware transcoding using the seperate go-vod instance. To make it work, adjust the following in the _Memories_ Admin GUI:\n  * enable \"Images\", \"HEIC\" and \"Videos\" under \"File Support\"\n  * define `/usr/bin/ffmpeg` / `/usr/bin/ffprobe` as ffmpeg / ffprobe path\n  * enable \"external transcoder\" and just write `go-vod:47788` under \"Connection address\" _(The Traefik DNS will resolve `go-vod` correctly within the docker proxy network)_.\n  * tick \"Enable acceleration with VA-API\" to \u003cu\u003eon\u003c/u\u003e. _(ignore the warning \"VA-API device not found\")_\n  * (note that I added the cronjob for preview generator in the mounted `crontab-www-data` file.)\n* **To enter the Nextcloud container** (e.g. to run `occ` commands), **run:**\n    ```bash\n    docker exec -itu www-data nextcloud bash\n    ```\n\n---\n\n## Notes \u0026 Tips\n\n\n* **for [ulogger](/roles/containers/templates/ulogger.yml.j2)**:\n  * The container is setup to use sqlite\n  * Before the first run: Either use a _named_ volume[^1] or start the container once with a bind mount other than `/data` to copy the container contents of `/data` onto the host to bootstrap the database.\n* The `makefile` supports shorthands for often used Ansible commands. For example, when only modifying `mpserver-maintenance.sh`, just run `make script` and it will only replace the script on the server.\n* The **`~/docker/docker-compose.yml`**, serves as a \"master\" docker-compose.yml file with all the activated services [included](https://docs.docker.com/compose/compose-file/14-include/) in the file. You can use it to execute actions on **all** containers at once just with one `docker-compose \u003cup|down|...\u003e` command.\n* \u003cdel\u003eMake sure to also re-deploy the maintenance script when adding/removing docker services in `group_vars` since the script will change according to the enabled services.\u003c/del\u003e[^2]\n* Enable `debug_ports_open: true` to open ports on the host bypassing the reverse proxy. This can be used for debugging, e.g. Traefik metrics and node_exporter is usually not accessible outside of `traefik-net`.\n\n[^1]:https://stackoverflow.com/q/65176940\n[^2]:Not needed anymore since we can use the new [include](https://docs.docker.com/compose/compose-file/14-include/) mechanism to generate one \"master\" docker-compose.yml.\n\n---\n\n## TODOs\n\n- [ ] Maintenance: Run all scrubbing tasks in parallel.\n- [x] Maintenance: Send Email when scrubbing starts, send another one when scrubbing has ended containing possible errors.\n- [x] Maintenance: Better log for transferred files during snapshots \u0026 backups with btrbk and rsync.\n- [x] Maintenance: Email formatting with `code` so that `\u003e\u003e` doesn't get interpreted as quote by some Email clients.\n- [x] Add overall downtime monitoring with alert when server comes back online (and possibly when it is down and monitoring is done from another host).\n- [ ] Improve Grafana dashboards.\n- [ ] Maintenance: Auto Reboot when Kernel has updated.\n- [ ] Add alertion system for suspicious events like access from a specific country or a DDOS attempt.\n- [x] Maintenance: Better system update / pacman / paru logging (do not log whole stdout).\n- [x] Docker-Compose: making local compose invocations work exchangeably with master-compose invocations (since `com.docker.compose.project` and `com.docker.compose.project.working_dir` Label don't match) (fixed using )\n\n\n---\n\n## Credits\n\nA large portion of the repository is inspired by [Alex Kretzschmar's infra repo](https://github.com/ironicbadger/infra) as well as [Wolfgang notthebee's infra repo](https://github.com/notthebee/infra). Also, both BTRFS maintenance and the system setup was largely inspired by [zilexa's Homeserver Guide](https://github.com/zilexa/Homeserver).\n\nThanks also goes to [Jeff Geerling's security role](https://github.com/geerlingguy/ansible-role-security) where I took almost all of my (not yet activated) SSH configuration.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmipro98%2Finfra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmipro98%2Finfra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmipro98%2Finfra/lists"}