{"id":15148286,"url":"https://github.com/yann39/self-hosted","last_synced_at":"2025-04-06T07:14:26.675Z","repository":{"id":221247935,"uuid":"714598718","full_name":"Yann39/self-hosted","owner":"Yann39","description":"Personal self-hosted infrastructure setup","archived":false,"fork":false,"pushed_at":"2025-03-01T21:32:03.000Z","size":7499,"stargazers_count":120,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-06T07:14:20.540Z","etag":null,"topics":["ackee","banana-pi","docker","docker-compose","homelab","homer","lychee","phpmyadmin","pi-hole","pihole","portainer","sablier","self-hosted","traefik","unbound","uptime-kuma","wireguard","wireguard-ui"],"latest_commit_sha":null,"homepage":"","language":"Dockerfile","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Yann39.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-05T10:53:49.000Z","updated_at":"2025-03-30T16:00:09.000Z","dependencies_parsed_at":"2024-02-24T18:26:31.354Z","dependency_job_id":"00db3e1b-c68a-42cb-8e1f-df404158fdde","html_url":"https://github.com/Yann39/self-hosted","commit_stats":{"total_commits":36,"total_committers":1,"mean_commits":36.0,"dds":0.0,"last_synced_commit":"8fef26fa14080e2ceaa9e0f28c14b6287b18b835"},"previous_names":["yann39/self-hosted"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yann39%2Fself-hosted","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yann39%2Fself-hosted/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yann39%2Fself-hosted/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Yann39%2Fself-hosted/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Yann39","download_url":"https://codeload.github.com/Yann39/self-hosted/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247445681,"owners_count":20939961,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ackee","banana-pi","docker","docker-compose","homelab","homer","lychee","phpmyadmin","pi-hole","pihole","portainer","sablier","self-hosted","traefik","unbound","uptime-kuma","wireguard","wireguard-ui"],"created_at":"2024-09-26T13:02:17.290Z","updated_at":"2025-04-06T07:14:26.642Z","avatar_url":"https://github.com/Yann39.png","language":"Dockerfile","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cimg src=\"images/header-text-only.svg\" alt=\"Header image\"/\u003e\n\n# Personal self-hosting guide\n\n![Static Badge](https://img.shields.io/badge/Version-1.2.0-2AAB92)\n![Static Badge](https://img.shields.io/badge/Last_update-02_Aug_2024-blue)\n![Static Badge](https://img.shields.io/badge/Free_\u0026_Open_source-GPL_V3-green)\n\nThis project describes my personal **self-hosted** infrastructure setup, running on a **Banana Pi M5** board.\n\nThis was meant to be just a reminder for me, but I wrote it as a guide, in case it might help someone.\n\nIt uses only **free** and **open source** software and hardware.\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cpicture\u003e\n \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"images/logo-open-source-initiative.svg\" height=\"128\"/\u003e\n \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"images/logo-open-source-initiative-black.svg\" height=\"128\"/\u003e\n \u003cimg alt=\"Open-source initiative logo\" src=\"images/logo-open-source-initiative-black.svg\" height=\"128\"/\u003e\n \u003c/picture\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-open-source-hardware.svg\" alt=\"Open source hardware logo\" height=\"128\"/\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e [!IMPORTANT]\n\u003e The content of this repository is provided \"as is\", with no guarantee that the information is complete or error-free.\n\u003e The techniques and tools discussed here come with inherent risks.\n\u003e The author takes absolutely no responsibility for possible consequences due to the use of the related software.\n\n# Table of Content\n\n1. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#overview\"\u003eOverview\u003c/a\u003e\u003c/summary\u003e\n\n 1. [Plan](#plan)\n 2. [Architecture](#architecture)\n\n \u003c/details\u003e\n2. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#banana-pi-m5-initial-setup\"\u003eBanana Pi M5 initial setup\u003c/a\u003e\u003c/summary\u003e\n\n 1. [Install Android on the eMMC storage](#install-android-on-the-emmc-storage)\n 2. [Format the eMMC storage](#format-the-emmc-storage)\n 3. [Install Armbian on the MicroSD](#install-armbian-on-the-microsd)\n 4. [Install Armbian on the eMMC storage](#install-armbian-on-the-emmc-storage)\n\n \u003c/details\u003e\n3. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#prepare-system\"\u003ePrepare system\u003c/a\u003e\u003c/summary\u003e\n\n 1. [User](#user)\n 2. [SSH access](#ssh-access)\n 3. [Cleaning](#cleaning)\n 4. [Directory structure](#directory-structure)\n 5. [Docker \u0026 Docker Compose](#docker--docker-compose)\n\n \u003c/details\u003e\n4. \u003cdetails open\u003e\n \u003csummary\u003e\u003ca href=\"#network-configuration\"\u003eNetwork configuration\u003c/a\u003e\u003c/summary\u003e\n\n 1. [IP settings](#ip-settings)\n 2. [Dynamic DNS](#dynamic-dns)\n 3. [Domain and subdomains](#domain-and-subdomains)\n 4. [Port forwarding](#port-forwarding)\n 5. [Reverse proxy](#reverse-proxy)\n 6. [VPN and ad-blocking](#vpn-and-ad-blocking)\n 7. [Test the network](#test-the-network)\n 8. [Network flow](#network-flow)\n\n \u003c/details\u003e\n5. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#install-services\"\u003eInstall services\u003c/a\u003e\u003c/summary\u003e\n\n 1. [Portainer](#portainer)\n 2. [PhpMyAdmin](#phpmyadmin)\n 3. [Homer](#homer)\n 4. [Dashdot](#dashdot)\n 5. [Uptime Kuma](#uptime-kuma)\n 6. [Ackee](#ackee)\n 7. [Lychee](#lychee)\n 8. [Defrag-life](#defrag-life)\n 9. [CCTeam](#ccteam)\n\n \u003c/details\u003e\n6. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#scale-to-zero-with-sablier\"\u003eScale to zero with Sablier\u003c/a\u003e\u003c/summary\u003e\n\n 1. [Install Sablier](#install-sablier)\n 2. [Install Traefik plugin](#install-traefik-plugin)\n 3. [Configure target applications](#configure-target-applications)\n\n \u003c/details\u003e\n7. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#backup\"\u003eBackup\u003c/a\u003e\u003c/summary\u003e\n\n 1. [Files](#files)\n 2. [Volumes](#volumes)\n 3. [Databases](#databases)\n\n \u003c/details\u003e\n8. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#contributing\"\u003eContributing\u003c/a\u003e\u003c/summary\u003e\n \u003c/details\u003e\n9. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#acknowledgments\"\u003eAcknowledgments\u003c/a\u003e\u003c/summary\u003e\n \u003c/details\u003e\n10. \u003cdetails\u003e\n \u003csummary\u003e\u003ca href=\"#license\"\u003eLicense\u003c/a\u003e\u003c/summary\u003e\n \u003c/details\u003e\n\n# Overview\n\n## Plan\n\nI started this project in late 2023 as a **home lab**, for learning, the goal was to have an environment :\n\n- **100% self-hosted** (privacy preserving, full control over data and software)\n- **Secure** (authentication, SSL/TLS, firewall, ad blocking, DDOS protection, rate limiting, custom DNS resolver, ...)\n- **Lightweight** (runs smoothly with minimal hardware and software requirements)\n- **Container-ready** (isolated, portable, scalable applications)\n- **Accessible** (some services accessible only locally, some only through VPN, some publicly on the internet)\n- **Supervised** (monitoring, alerting, tracking, and backup tools)\n\nThese are the tools we are going to run :\n\n| Logo | Name | Repository | Description |\n|:---------------------------------------------------------------------------------:|----------------|---------------------------------------------|------------------------------------------------------|\n| \u003cimg src=\"images/logo-docker.svg\" alt=\"Docker logo\" height=\"24\"/\u003e | Docker | https://github.com/docker | Help to build, share, and run container applications |\n| \u003cimg src=\"images/logo-docker-compose.png\" alt=\"Docker Compose logo\" height=\"38\"/\u003e | Docker Compose | https://github.com/docker/compose | Run multi-container applications with Docker |\n| \u003cimg src=\"images/logo-portainer.svg\" alt=\"Portainer logo\" height=\"32\"/\u003e | Portainer | https://github.com/portainer/portainer | Management platform for containerized applications |\n| \u003cimg src=\"images/logo-sablier.svg\" alt=\"Sablier logo\" height=\"32\"/\u003e | Sablier | https://github.com/acouvreur/sablier | Workload scaling on demand |\n| \u003cimg src=\"images/logo-traefik.svg\" alt=\"Traefik logo\" height=\"35\"/\u003e | Traefik | https://github.com/traefik/traefik | Modern HTTP reverse proxy and load balancer |\n| \u003cimg src=\"images/logo-wireguard.svg\" alt=\"Wireguard logo\" height=\"30\"/\u003e | Wireguard | https://github.com/WireGuard | Simple yet fast and modern VPN |\n| \u003cimg src=\"images/logo-wireguard.svg\" alt=\"Wireguard logo\" height=\"30\"/\u003e | Wireguard UI | https://github.com/ngoduykhanh/wireguard-ui | Web user interface to manage WireGuard setup |\n| \u003cimg src=\"images/logo-pihole.svg\" alt=\"Pi-hole logo\" height=\"34\"/\u003e | Pi-hole | https://github.com/pi-hole/pi-hole | Network-wide ad blocking |\n| \u003cimg src=\"images/logo-unbound.svg\" alt=\"Unbound logo\" height=\"32\"/\u003e | Unbound | https://github.com/NLnetLabs/unbound | Validating, recursive, and caching DNS resolver |\n| \u003cimg src=\"images/logo-uptime-kuma.svg\" alt=\"Uptime Kuma logo\" height=\"34\"/\u003e | Uptime Kuma | https://github.com/louislam/uptime-kuma | Easy-to-use self-hosted monitoring tool |\n| \u003cimg src=\"images/logo-homer.png\" alt=\"Homer logo\" height=\"30\"/\u003e | Homer | https://github.com/bastienwirtz/homer | Static application dashboard |\n| \u003cimg src=\"images/logo-dashdot.png\" alt=\"Dashdot logo\" height=\"32\"/\u003e | Dashdot | https://github.com/MauriceNino/dashdot | Minimal server dashboard and monitoring |\n| \u003cimg src=\"images/logo-ackee.png\" alt=\"Ackee logo\" height=\"32\"/\u003e | Ackee | https://github.com/electerious/Ackee | Analytics tool that cares about privacy |\n| \u003cimg src=\"images/logo-lychee.png\" alt=\"Lychee logo\" height=\"32\"/\u003e | Lychee | https://github.com/LycheeOrg/Lychee | Free photo-management tool |\n| \u003cimg src=\"images/logo-phpmyadmin.svg\" alt=\"PhpMyAdmin logo\" height=\"32\"/\u003e | PhpMyAdmin | https://github.com/phpmyadmin/phpmyadmin | Web user interface to manage MySQL databases |\n\nAnd also some personal applications :\n\n- My first **PHP** / **MySQL** website from the early 2000's ! : https://github.com/Yann39/defrag-life\n- A **GraphQL API** (**Java** / **Spring Boot**) for one of my **Flutter** mobile applications : https://github.com/Yann39/ccteam-graphql\n\nAll of this runs on a single **Banana Pi M5 board** ! With the following specifications :\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/banana-pi-front.png\" alt=\"Banana board front view\" height=\"172\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cul\u003e\n \u003cli\u003eAmlogic S905X3 64-bit Quad core Cortex-A55 (2.0 GHz)\u003c/li\u003e\n \u003cli\u003eGPU Mali-G31 MP2\u003c/li\u003e\n \u003cli\u003e4GB LPDDR4\u003c/li\u003e\n \u003cli\u003e16GB eMMC flash\u003c/li\u003e\n \u003cli\u003e1 GbE ethernet\u003c/li\u003e\n \u003cli\u003e4 x USB 3.0\u003c/li\u003e\n \u003c/ul\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e [!NOTE]\n\u003e This hardware is obviously not designed for high loads,\n\u003e I only have a few users on my public applications,\n\u003e of course if you need to handle more load you might consider a better machine.\n\nIt should work on many other **ARM** boards such as the **Raspberry Pi**,\nand also on **x86**-based device by just using the right Docker image.\n\n## Architecture\n\nHere is a chart representing the global network \"architecture\" we are going to set up, simplified with only the most relevant services.\nSee [Network flow](#network-flow) for more detailed schemas.\n\nThis architecture allows exposing applications to the internet while restricting access to some of them only through **VPN** or from the local network.\nIt's up to you to choose the accessibility level you need for each service, you may want some to be accessible only from your local network,\nsome only via VPN, and others to anyone from the internet.\n\n```mermaid\nflowchart TB\n style HOSTING_PROVIDER fill:#4d683b,color:#fff\n style DDNS_PROVIDER fill:#69587b,color:#fff\n style INTERNET_SERVICE_PROVIDER fill:#205566,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n style CONTAINER_ENGINE fill:#664343,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style PIHOLE_CONTAINER fill:#663535,color:#fff\n style SABLIER_CONTAINER fill:#663535,color:#fff\n style UNBOUND_CONTAINER fill:#663535,color:#fff\n style MYAPP_CONTAINER fill:#663535,color:#fff\n style WIREGUARD_CONTAINER fill:#663535,color:#fff\n style TRAEFIK_ROUTER fill:#806030,color:#fff\n style TRAEFIK_MIDDLEWARE fill:#806030,color:#fff\n style VPN_CLIENT fill:#105040,color:#fff\n style PIHOLE_DNS_RECORDS fill:#806030,color:#fff\n DOMAIN(example.com)\n SUBDOMAIN_WIREGUARD(wireguard.example.com)\n SUBDOMAIN_MYAPP(myapp.example.com)\n DDNS(myddns.ddns.net)\n ROUTER[public IP]\n ROUTER_PORT80{{80/tcp}}\n ROUTER_PORT443{{443/tcp}}\n ROUTER_PORT51820{{51820/udp}}\n DOCKER_WIREGUARD_PORT51820{{51820/udp}}\n DOCKER_MYAPP_PORT5000{{5000/tcp}}\n DOCKER_PIHOLE_PORT80{{80/tcp}}\n DOCKER_PIHOLE_PORT53{{53/udp}}\n DOCKER_TRAEFIK_PORT443{{433/tcp}}\n DOCKER_TRAEFIK_PORT80{{80/tcp}}\n DOCKER_TRAEFIK_PORT8080{{8080/tcp}}\n DOCKER_UNBOUND_PORT53{{53/udp}}\n DOCKER_SABLIER_PORT10000{{10000/tcp}}\n TRAEFIK_ROUTER_MYAPP(myapp\u003cbr/\u003e.example.com)\n TRAEFIK_ROUTER_PIHOLE(pihole\u003cbr/\u003e.example.com)\n TRAEFIK_ROUTER_TRAEFIK(traefik\u003cbr/\u003e.example.com)\n ROOT_DNS_SERVERS[Root DNS servers]\n DNS_ISP[DNS 1 \u0026 2]\n DOCKER_PIHOLE_DNS[DNS 1 \u0026 2]\n PIHOLE_DNS_PIHOLE[pihole\u003cbr/\u003e.example.com]\n PIHOLE_DNS_TRAEFIK[traefik\u003cbr/\u003e.example.com]\n\n subgraph VPN_CLIENT[VPN CLIENT]\n WIREGUARD_CLIENT_ENDPOINT[Endpoint]\n WIREGUARD_CLIENT_DNS[DNS]\n end\n\n subgraph HOSTING_PROVIDER[DOMAIN NAME REGISTRAR]\n DOMAIN --\u003e|subdomain| SUBDOMAIN_MYAPP\n DOMAIN --\u003e|subdomain| SUBDOMAIN_WIREGUARD\n end\n\n subgraph DDNS_PROVIDER[DYNAMIC DNS PROVIDER]\n SUBDOMAIN_MYAPP ---\u003e|CNAME| DDNS\n SUBDOMAIN_WIREGUARD ---\u003e|CNAME| DDNS\n end\n\n subgraph INTERNET_SERVICE_PROVIDER[INTERNET SERVICE PROVIDER]\n DDNS ---\u003e|DynDNS| ROUTER\n ROUTER --\u003e ROUTER_PORT80\n ROUTER --\u003e ROUTER_PORT443\n ROUTER --\u003e ROUTER_PORT51820\n DNS_ISP\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI M5]\n subgraph CONTAINER_ENGINE[DOCKER]\n\n subgraph TRAEFIK_CONTAINER[TRAEFIK CONTAINER]\n subgraph TRAEFIK_ROUTER[TRAEFIK HTTP ROUTER]\n TRAEFIK_ROUTER_TRAEFIK\n TRAEFIK_ROUTER_MYAPP\n TRAEFIK_ROUTER_PIHOLE\n end\n subgraph TRAEFIK_MIDDLEWARE[TRAEFIK MIDDLEWARE]\n REDIRECT(HTTPS redirect)\n IP_WHITELISTING(IP whitelist)\n SABLIER(Sablier dynamic)\n BASIC_AUTH(Basic auth)\n end\n DOCKER_TRAEFIK_PORT80\n DOCKER_TRAEFIK_PORT443\n DOCKER_TRAEFIK_PORT8080\n end\n\n subgraph SABLIER_CONTAINER[SABLIER CONTAINER]\n DOCKER_SABLIER_PORT10000\n WAITING_PAGE(Waiting page)\n end\n\n subgraph PIHOLE_CONTAINER[PIHOLE CONTAINER]\n subgraph PIHOLE_DNS_RECORDS[LOCAL DNS RECORDS]\n PIHOLE_DNS_TRAEFIK\n PIHOLE_DNS_PIHOLE\n end\n DOCKER_PIHOLE_PORT53\n DOCKER_PIHOLE_PORT80\n DOCKER_PIHOLE_DNS\n end\n\n subgraph WIREGUARD_CONTAINER[WIREGUARD CONTAINER]\n DOCKER_WIREGUARD_PORT51820\n end\n\n subgraph MYAPP_CONTAINER[MYAPP CONTAINER]\n DOCKER_MYAPP_PORT5000\n end\n\n subgraph UNBOUND_CONTAINER[UNBOUND CONTAINER]\n DOCKER_UNBOUND_PORT53\n end\n\n end\n\n end\n\n WIREGUARD_CLIENT_ENDPOINT ---\u003e SUBDOMAIN_WIREGUARD\n WIREGUARD_CLIENT_DNS --\u003e|Pi - Hole internal IP| DOCKER_PIHOLE_PORT53\n ROUTER_PORT51820 --\u003e|port forward| DOCKER_WIREGUARD_PORT51820\n ROUTER_PORT443 ------\u003e|port forward| DOCKER_TRAEFIK_PORT443\n ROUTER_PORT80 --\u003e|port forward| DOCKER_TRAEFIK_PORT80\n DNS_ISP --\u003e|Banana Pi M5 static IP| DOCKER_PIHOLE_PORT53\n PIHOLE_DNS_TRAEFIK ---\u003e|Banana Pi internal IP| DOCKER_TRAEFIK_PORT443\n PIHOLE_DNS_PIHOLE ---\u003e|Banana Pi internal IP| DOCKER_TRAEFIK_PORT443\n DOCKER_TRAEFIK_PORT443 --\u003e TRAEFIK_ROUTER\n DOCKER_TRAEFIK_PORT80 --\u003e TRAEFIK_ROUTER\n TRAEFIK_ROUTER_MYAPP --\u003e REDIRECT\n TRAEFIK_ROUTER_PIHOLE --\u003e REDIRECT\n TRAEFIK_ROUTER_TRAEFIK --\u003e|Dashboard / API| REDIRECT\n IP_WHITELISTING --\u003e BASIC_AUTH\n IP_WHITELISTING --\u003e DOCKER_PIHOLE_PORT80\n REDIRECT ----\u003e SABLIER\n SABLIER \u003c-..-\u003e|return status| DOCKER_SABLIER_PORT10000\n SABLIER ---\u003e|not ready| WAITING_PAGE\n SABLIER ---\u003e|ready| DOCKER_MYAPP_PORT5000\n REDIRECT --\u003e IP_WHITELISTING\n DOCKER_SABLIER_PORT10000 \u003c-.-\u003e|check status| DOCKER_MYAPP_PORT5000\n BASIC_AUTH --\u003e DOCKER_TRAEFIK_PORT8080\n DOCKER_PIHOLE_DNS ---\u003e DOCKER_UNBOUND_PORT53\n UNBOUND_CONTAINER \u003c----\u003e ROOT_DNS_SERVERS\n```\n\nBasically all services will be accessible via dedicated subdomains which will point to our local network, either through **dynamic DNS** or through **local DNS records**,\nthen a **reverse proxy** will be responsible for routing the requests to the right application running in **Docker** containers.\n\nWe can make the **ISP upstream DNS** (from **router** configuration) point to the Banana Pi **IP address**,\nso that we reroute the entire Internet traffic through **Pi-hole** and thus take advantage of its benefits.\n\nIn this example **Traefik** (_traefik.example.com_) and **Pi-Hole** (_pihole.example.com_) are only accessible\nthrough VPN and from the local network thanks to local DNS records and IP whitelisting,\nwhile **MyApp** (_myapp.example.com_) is also accessible from the internet publicly.\n\nThe Traefik dashboard is protected by **basic authentication**, and the _MyApp_ container is started/stopped on-demand through **Sablier**.\n\nYou will find more details on how all this has been implemented later in this guide.\n\n# Banana Pi M5 initial setup\n\n\u003cimg src=\"images/logo-banana-pi.svg\" alt=\"Banana Pi logo\" height=\"120\"/\u003e\n\nBy default, there is no **operating system** installed on the Banana Pi.\nWe have to install a system either on the **MicroSD** card or on the **eMMC** storage.\nThe eMMC storage offers better performance but is limited to **16Gb**, it should be enough for our needs though.\n\nI will describe below the procedures to install a system on the MicroSD card and on the eMMC storage.\n\n## Install Android on the eMMC storage\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-android.svg\" alt=\"Android logo\" height=\"64\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-emmc.svg\" alt=\"Armbian logo\" height=\"36\"/\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\n\u003e [!NOTE]\n\u003e This is the first thing I did to test it out, but actually installing **Android** is optional, you can directly [format the eMMC storage](#format-the-emmc-storage)\n\u003e then [install Armbian on the eMMC storage](#install-armbian-on-the-emmc-storage), you will still need to execute some of the steps described below though.\n\nFrom your machine (**Windows 11** in my case) :\n\n- Download **Amlogic USB Burning Tool** v3.1.0 (v3.2.0 seem not to work, error is raised while loading the image)\n- Download latest **Android** image for Banana Pi M5 : _2023-03-01-bpi-m5-m2pro-tablet-android9.img.zip_\n- Extract it to get the _2023-03-01-bpi-m5-m2pro-tablet-android9.img_ file\n- Execute Amlogic USB Burning Tool as Administrator\n- Load Android image from \"Setting -\u003e Load img\" menu\n- Press **SW4** button on the Banana Pi for 2/3s (don't know why and if it is really necessary, but it is specified in the doc)\n- Connect the USB cable from PC to Banana Pi\n- Device should be detected in Amlogic USB Burning Tool, just click \"Start\" and wait for the operation to complete\n \u003cimg src=\"images/screen-usb-burning-tool.png\" alt=\"USB Burning Tool screenshot\"/\u003e\n- Unplug the USB cable from Banana Pi and PC\n- Plug in the USB-C power cable to the Banana Pi, it should boot on Android\n\n## Format the eMMC storage\n\n\u003cimg src=\"images/logo-emmc2.png\" alt=\"eMMC logo\" height=\"74\" /\u003e\n\nYou need to format the eMMC storage to be able to install a new system. To do that :\n\n- Execute the same steps as above ([Install Android on the eMMC storage](#install-android-on-the-emmc-storage)) but unplug the USB cable during the \"Formatting\" step (not too early\n and not too\n late, had to do it multiple times until it worked)\n- Plug in the USB-C power cable to the Banana Pi, it should boot on the MicroSD card, indeed **the Banana Pi will boot on the MicroSD card only if the eMMC storage is empty**\n\n## Install Armbian on the MicroSD\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-armbian.png\" alt=\"Armbian logo\" height=\"42\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cpicture\u003e\n \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"images/logo-microsd.svg\" height=\"42\"/\u003e\n \u003csource media=\"(prefers-color-scheme: light)\" srcset=\"images/logo-microsd-black.svg\" height=\"42\"/\u003e\n \u003cimg alt=\"MicroSD logo\" src=\"images/logo-microsd-black.svg\" height=\"42\"/\u003e\n \u003c/picture\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nI decided to use **Armbian** as operating system, Armbian is a **Linux** distribution designed for **ARM** development boards.\nUnlike Raspbian, Armbian focuses on unifying the experience across many ARM single-board computers.\n\nI first installed it on the **MicroSD** card, then used it to burn the Armbian image into the eMMC storage later\n(see [install Armbian on the eMMC storage](#install-armbian-on-the-emmc-storage)).\n\nFrom your machine (**Windows 11** in my case) :\n\n- Download latest **Armbian** image for Banana Pi M5, choose the CLI or the GUI image depending on your preference :\n - With GUI : _Armbian_23.02.2_Bananapim5_bullseye_current_6.1.11_gnome_desktop.img.xz_\n - Without GUI (CLI) : _Armbian_23.02.2_Bananapim5_bullseye_current_6.1.11.img.xz_\n- Extract it to get the _img_ file\n- Connect the MiroSD card to the PC\n- Download **Rufus** (3.21 when writing this) or equivalent software to be able to write the image to the MicroSD card\n- Simply select the image in **Rufus** and write it to the MicroSD card with the default proposed options\n- Insert the MicroSD card into the Banana Pi and plug in the USB power cable, this should boot on Armbian on the MicroSD card\n\n## Install Armbian on the eMMC storage\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-armbian.png\" alt=\"Armbian logo\" height=\"42\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-emmc.svg\" alt=\"Armbian logo\" height=\"36\"/\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nWe will install Armbian in the eMMC storage, this setup will offer the best performances.\n\n- Plug in the USB-C power cable of the Banana Pi M5 to boot on Armbian in the MicroSD card\n- Put the Armbian image in the Banana Pi MicroSD card storage (through USB key or network or whatever), for example in _/home/me/Documents_\n- Run `fdisk -l` command to identify the **eMMC** path, it should be something like _/dev/mmcblk1_\n- Burn the image to the eMMC storage by running the command :\n ```shell\n sudo dd if=Armbian_23.02.2_Bananapim5_bullseye_current_6.1.11_gnome_desktop.img of=/dev/mmcblk1 bs=10MB\n ```\n- Remove the MicroSD card and reboot the Banana Pi M5, it should boot on Armbian on the eMMC storage\n\n# Prepare system\n\n## User\n\nWhen installing **Armbian**, you should have been asked to create a **regular user account** that is **sudo** enabled.\nWe will simply use that user for the whole guide.\n\nFor security reasons, do not use the `root` user directly.\nIf you run a program as root and a security flaw is exploited, the attacker has access to the whole system without restriction.\nUsing a regular user, even with sudo enabled, will require running `sudo` and will still prompt for the account password as an additional security step.\nIt is also safer in case you unintentionally issue a command that could hurt the system (like deleting system files, etc.).\n\n\u003e [!NOTE]\n\u003e We may also create specific users inside **Docker containers** for some applications, specially when creating our own **Dockerfile**,\n\u003e but we'll clarify then whether additional permissions need to be added in case they need access to the local filesystem through a **bind mount**.\n\n## SSH access\n\nGenerally, you'll want to leave your board in a cool, quiet corner,\nrather than letting it land around in your feet and having to connect a keyboard/mouse/screen every time you want to access it.\n\nA solution is simply to access it as a remote computer via **SSH**, from your main computer.\n\nIn the normal **Armbian** images, SSH is enabled by default, so there is no additional configuration to do.\n\nSimply use the `ssh` command to establish a secure and authenticated SSH connection to the board :\n\n```shell\nssh me@bananapim5\n```\n\nEnter your password then you are ready to go !\n\n\u003cimg src=\"images/screen-ssh-bpi.png\" alt=\"Bananapi header after SSH connection\"/\u003e\n\nYou can also use your preferred **SSH client** (on Windows I usually use **MobaXterm**).\n\nUnless you want to be able to do some operations from outside your local network, there is no need to open the SSH port to the internet.\nIf you do so consider using it behind a VPN (even if SSH itself is very secure).\n\n## Cleaning\n\n**Armbian** come with default installed software that we will not use. If, like me, you chose the GUI image, let's remove some packages to save disk space.\n\n1. Upgrade packages :\n\n ```shell\n sudo apt update\n sudo apt upgrade\n ```\n\n2. Remove packages we don't need :\n\n ```shell\n sudo apt purge --auto-remove gimp\n sudo apt purge --auto-remove hexchat\n sudo apt purge qbittorrent\n sudo apt purge telegram-desktop\n sudo apt purge pithos\n sudo apt purge pidgin\n sudo apt purge thunderbird\n sudo apt purge --auto-remove geany\n sudo apt purge meld\n sudo apt purge libreoffice*\n sudo apt purge --auto-remove mc\n sudo apt purge --auto-remove transmission\n sudo apt purge transmission-remote-gtk\n sudo apt remove kazam\n sudo apt remove remmina\n sudo apt remove codium\n sudo apt remove mpv\n sudo apt remove sysstat\n sudo apt autoremove\n sudo apt autoclean\n ```\n\nThen we can also remove **unneeded locales** to save some more space, using the `localepurge` tool. `localepurge` is a small script to recover disk space wasted for unneeded locale\nfiles and localized man pages.\n\nInstall the package :\n\n```shell\nsudo apt install localepurge\n```\n\nThis will automatically run the script to allow selecting the locales we want to keep, i.e. I selected :\n\n```\nen\nen_US.UTF-8\nfr\nfr_CH.UTF-8\n```\n\nAny locale you have not selected will be purged.\n\nIf you need to run it again, execute :\n\n```shell\nsudo dpkg-reconfigure localepurge\n```\n\nYou can also use the **BleachBit** utility, installed by default on Armbian, to clean the system, i.e. I run it with following options checked :\n\n- `autoclean` : delete obsolete files\n- `autoremove` : delete obsolete files\n- `clean` : delete the APT cache\n- `package lists` : delete the package list cache\n- `journald` :\n - `clean` : clean old system journals\n- `system`\n - `broken desktop files` : delete broken application menu entries and file associations\n - `cache` : delete system cache\n - `localizations` : delete files for unwanted languages\n - `rotated logs` : delete old system logs\n - `temporary files` : delete the temporary files\n - `trash` : empty the trash\n\nAll of this should have saved some megabytes and unnecessary disk I/O.\n\n## Directory structure\n\nWe will place every application configuration into the _/opt/apps_ directory, as follows :\n\n ```\n /\n |- opt\n |- apps\n |- traefik\n |- portainer\n |- phpmyadmin\n |- dashdot\n |- ...\n ```\n\nUsually this directory (_/opt_) is reserved for any software and packages that are not part of the default installation,\nbut feel free to choose another location.\n\nYou can already create the directory :\n\n```shell\nsudo mkdir /opt/apps\n```\n\nWe will create the subdirectories associated with each application when we install them.\n\n## Docker \u0026 Docker Compose\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-docker.svg\" alt=\"Docker logo\" height=\"128\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-docker-compose.png\" alt=\"Docker logo\" height=\"148\"/\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nWe will use **Docker** to containerize and run our different applications.\n\nDocker enables to separate applications from the infrastructure, it provides the ability to package and run an application in an isolated environment called a **container**.\nContainers contain everything needed to run the application, so you don't need to rely on what's installed on the host.\n\nDocker provides an installation script, just run it :\n\n```shell\ncurl -fsSL https://get.docker.com -o get-docker.sh\nsudo sh get-docker.sh\nsudo rm get-docker.sh\nsudo docker version\n```\n\n\u003e [!NOTE]\n\u003e You can use https://get.docker.com/rootless to install it in rootless mode (run the Docker daemon as a non-root user)\n\u003e to mitigate potential vulnerabilities in the daemon and the container runtime.\n\nThen also install **Docker-Compose**, so we can define and run multi-container Docker applications :\n\n```shell\nsudo apt install docker-compose -y\nsudo docker-compose version\n```\n\nThat's it :\n\n```shell\nsudo docker info\n```\n\n```console\nClient:\nContext: default\nDebug Mode: false\nPlugins:\nbuildx: Docker Buildx (Docker Inc.)\nVersion: v0.10.4\nPath: /usr/libexec/docker/cli-plugins/docker-buildx\ncompose: Docker Compose (Docker Inc.)\nVersion: v2.17.3\nPath: /usr/libexec/docker/cli-plugins/docker-compose\n```\n\n\u003e [!NOTE]\n\u003e In this guide I systematically use latest images (`:latest`tag), but usually you better want to avoid using `:latest` tags in production.\n\u003e Anyway if you use `latest` tags and want to update an image in the future, simply pull it again and rerun your container / compose file, i.e. :\n\u003e\n\u003e ```shell\n\u003e sudo docker-compose pull\n\u003e sudo docker-compose up -d\n\u003e ```\n\u003e\n\u003e Then remove any old images.\n\n# Network configuration\n\nBefore installing our services, we need to configure the network, so we can reach our applications from different locations.\n\nThe idea is to have :\n\n- A main **domain** name\n- A **subdomain** name for each application that must be reachable from the internet\n- A **dynamic DNS** name to avoid having to use a **static** public IP address\n- A **Traefik** reverse proxy to handle HTTP request that will be port forwarded to the applications\n\nFor services that will not be accessible to the internet, we will use **Pi-Hole**’s ability to manage **local DNS records**\n(each record will point to Banana Pi's internal IP address) so that they are also reachable using a subdomain name.\n\nHere is an overview of the route for each case, when a client request _myapp.example.com_ :\n\n:small_blue_diamond: Internet access :\n\n```mermaid\nflowchart LR\n style HOSTING_PROVIDER fill:#4d683b,color:#fff\n style DDNS_PROVIDER fill:#69587b,color:#fff\n style INTERNET_SERVICE_PROVIDER fill:#205566,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style APPLICATION fill:#663535,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n CLIENT((Client))\n SUBDOMAIN_MYAPP(myapp\u003cbr/\u003e.example.com)\n DDNS(myddns\u003cbr/\u003e.ddns.net)\n ROUTER[public IP]\n ROUTER_PORT{{port}}\n DOCKER_TRAEFIK_PORT{{port}}\n APPLICATION_PORT{{port}}\n\n subgraph HOSTING_PROVIDER[DOMAIN NAME REGISTRAR]\n SUBDOMAIN_MYAPP\n end\n\n subgraph DDNS_PROVIDER[DYNAMIC DNS PROVIDER]\n DDNS\n end\n\n subgraph INTERNET_SERVICE_PROVIDER[INTERNET SERVICE PROVIDER]\n ROUTER\n ROUTER_PORT\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI]\n subgraph TRAEFIK_CONTAINER[TRAEFIK]\n DOCKER_TRAEFIK_PORT\n end\n\n subgraph APPLICATION[APPLICATION]\n APPLICATION_PORT\n end\n end\n\n CLIENT --\u003e SUBDOMAIN_MYAPP\n SUBDOMAIN_MYAPP --\u003e|CNAME| DDNS\n DDNS --\u003e|DynDNS| ROUTER\n ROUTER --\u003e ROUTER_PORT\n ROUTER_PORT --\u003e|port forward| DOCKER_TRAEFIK_PORT\n DOCKER_TRAEFIK_PORT --\u003e|HTTP router| APPLICATION_PORT\n```\n\n:small_blue_diamond: VPN access :\n\n```mermaid\nflowchart LR\n style VPN fill:#4d683b,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style PI_HOLE fill:#663535,color:#fff\n style APPLICATION fill:#663535,color:#fff\n style WIREGUARD fill:#663535,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n CLIENT((Client))\n VPN_CLIENT(DNS)\n VPN_ENDPOINT(Endpoint)\n PIHOLE_DNS_MYAPP(myapp\u003cbr/\u003e.example.com)\n DOCKER_TRAEFIK_PORT{{port}}\n APPLICATION_PORT{{port}}\n WIREGUARD_PORT{{port}}\n PIHOLE_DNS{{port}}\n\n subgraph VPN[VPN]\n VPN_CLIENT\n VPN_ENDPOINT\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI]\n subgraph PI_HOLE[PI-HOLE]\n PIHOLE_DNS\n PIHOLE_DNS_MYAPP\n end\n\n subgraph TRAEFIK_CONTAINER[TRAEFIK]\n DOCKER_TRAEFIK_PORT\n end\n\n subgraph APPLICATION[APPLICATION]\n APPLICATION_PORT\n end\n\n subgraph WIREGUARD[WIREGUARD]\n WIREGUARD_PORT\n end\n end\n\n VPN_ENDPOINT --\u003e WIREGUARD_PORT\n CLIENT --\u003e VPN_CLIENT\n VPN_CLIENT --\u003e PIHOLE_DNS\n PIHOLE_DNS_MYAPP ---\u003e|A| TRAEFIK_CONTAINER\n DOCKER_TRAEFIK_PORT --\u003e|HTTP router| APPLICATION_PORT\n```\n\n:small_blue_diamond: Local network access :\n\n```mermaid\nflowchart LR\n style INTERNET_SERVICE_PROVIDER fill:#205566,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style PI_HOLE fill:#663535,color:#fff\n style APPLICATION fill:#663535,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n CLIENT((Client))\n ISP_DNS(DNS)\n PIHOLE_DNS_MYAPP(myapp\u003cbr/\u003e.example.com)\n DOCKER_TRAEFIK_PORT{{port}}\n APPLICATION_PORT{{port}}\n PIHOLE_DNS{{port}}\n\n subgraph INTERNET_SERVICE_PROVIDER[INTERNET SERVICE PROVIDER]\n ISP_DNS\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI]\n subgraph PI_HOLE[PI-HOLE]\n PIHOLE_DNS\n PIHOLE_DNS_MYAPP\n end\n\n subgraph TRAEFIK_CONTAINER[TRAEFIK]\n DOCKER_TRAEFIK_PORT\n end\n\n subgraph APPLICATION[APPLICATION]\n APPLICATION_PORT\n end\n\n end\n\n CLIENT ---\u003e ISP_DNS\n ISP_DNS ---\u003e PIHOLE_DNS\n PIHOLE_DNS_MYAPP ---\u003e|A| TRAEFIK_CONTAINER\n DOCKER_TRAEFIK_PORT --\u003e|HTTP router| APPLICATION_PORT\n```\n\n\u003e [!NOTE]\n\u003e My router offers all the required features (DHCP server, DNS server, port forwarding, dynDNS, etc.) for the steps described below.\n\u003e Most of the routers also have those features (they rarely purely route packets),\n\u003e but if this is not your case, you may have to perform **double NAT** to allow more advanced configurations.\n\u003e I obviously cannot go through the configuration specific to each router.\n\n## IP settings\n\nThe following changes to the IP settings are required if you want all your internet traffic to be redirected to your Banana Pi board so that\nevery request goes through **Pi-Hole** and use the custom **DNS resolver** (**Unbound**) :\n\n- Assign a **static IP address** to the Banana Pi board, for example `192.168.0.17` (I have local **DHCP** enabled)\n- Set **DNS** (primary and secondary) manually, to point to the Banana Pi board address set up above (`192.168.0.17`)\n\nOf course Pi-Hole container have to expose port **53** to receive incoming DNS requests. Refer to [Pi-hole](#pi-hole) setup for more details.\n\n## Dynamic DNS\n\nWhen connecting from outside our network (from the internet), we need to know the **public IP address** of our router to connect to.\nBut unless we have a **static** public IP (not necessarily the safest option), we are getting dynamically-assigned public IP addresses (via **DHCP**),\nso we would need to update the configuration everytime the IP changes, which is very uncomfortable.\n\nFortunately we can register a **dynamic host record** (DynDNS), and configure it in our router configuration so that when the public IP address changes, a call is made to the\nDynDNS service provider to update the record. That way our network will always be reachable from the internet via the **DynDNS** record no matter the IP address.\n\nWell, simply register a **dynamic DNS** hostname from a provider (there are free ones), for example **No-IP**, **DuckDNS**, etc. :\n\n- hostname : `myddns.ddns.net`\n- IP / target : _internet box external IP (public IP)_\n- type : `A`\n\nThen activate **DynDNS** on the router :\n\n- Service provider : `No-IP` (adapt to your provider)\n- Hostname : `myddns.ddns.net`\n- Username : _xxxxxxxx_\n- Password : _xxxxxxxx_\n\nThe IP will be updated automatically when a change will be detected.\n\n\u003e [!NOTE]\n\u003e Your ISP may only support some dynamic DNS provider that can be configured in the router, so you may want to pick one that is supported natively,\n\u003e else you will have to set up an **update client** that will be responsible to regularly check for IP change.\n\n## Domain and subdomains\n\nYou will need to buy a **domain** from you preferred domain provider, for this guide I will use `example.com`.\n\n\u003e [!IMPORTANT]\n\u003e I advise you to also subscribe to a **domain privacy** option in order to hide you personal data.\n\u003e Domain Privacy protects the contact information of the owner of a domain name in the **WHOIS directory**.\n\u003e Normally, this public database is used to verify the availability of a domain name and who it belongs to,\n\u003e but marketing companies and scammers can also exploit it for other purposes, like sending spam or identity theft.\n\nYou can check the information that are available publicly about your domain using the `whois` command :\n\n```shell\nwhois example.com\n```\n\nRight, we will then use **subdomains** to locate each service as a separate website to avoid having to buy a new domain name for each.\nA subdomain is simply a prefix added to the original domain name, it functions as a separate website from its domain.\n\nSo, let's create **subdomains** from the domain name registrar settings, for every service to be exposed on the internet :\n\n- `wireguard.example.com` : To access the [WireGuard](#wireguard) server\n- `quake.example.com` : To access the [Defrag-life](#defrag-life) website\n- `lychee.example.com` : To access the [Lychee](#lychee) website\n- `ccteam.example.com` : To access the [CCTeam](#ccteam) APIs\n\nAnd add corresponding **CNAME records** to point to the dynamic DNS `myddns.ddns.net` :\n\n- `CNAME\twireguard\t myddns.ddns.net`\n- `CNAME\tquake\t myddns.ddns.net`\n- `CNAME\tlychee\t myddns.ddns.net`\n- `CNAME\tccteam\t myddns.ddns.net`\n\nA **CNAME record** is just a records which points a name to another name instead of pointing to an IP address (like **A** records).\n\n\u003e [!NOTE]\n\u003e Services that will only be accessible from the local network or through VPN do **not** need to have a subdomain defined at this level.\n\u003e We will use Pi-Hole's **local DNS records** for that. See [Pi-Hole configuration](#pi-hole).\n\u003e\n\u003e However, while the VPN stuff is fully functional and to be able to do the configuration easily from your client machine,\n\u003e you may want to temporarily create subdomains and add CNAME records for the following subdomains\n\u003e (also remove the IP whitelisting middleware in the corresponding service configuration), else you will be blocked by IP whitelisting :\n\u003e\n\u003e - `wireguard-ui.example.com` : To configure the WireGuard VPN and create clients\n\u003e - `portainer.example.com` : To manage Docker containers (start/stop, check logs, etc.)\n\u003e - `pihole.example.com` : To configure the local DNS\n\n## Port forwarding\n\nFor our services to be reachable from the internet, we need to **forward incoming requests** to our Banana Pi board so that they will be handled by our **Traefik** reverse proxy.\nThis can be done through **port forwarding**.\n\nPort forwarding directs the **router** to send any incoming data from the internet to a specified device on the network.\nIt is safe to forward ports on your router as long as you have a **reverse proxy** or a **firewall** running in between.\n\n### Allow access without VPN\n\nIf you decide that at least one of the applications must be reachable from the outside directly through **HTTP** or **HTTPS** without requiring a **VPN**,\nthen simply port forward the related **TCP** ports to the Banana Pi.\n\nGo to your router configuration and add a **port forward rule** for the **TCP** port `80` :\n\n- Name : `Traefik`\n- Input port : `80`\n- Target port : `80`\n- Device : `bananapim5`\n- Protocol : `TCP`\n\nand `443` :\n\n- Name : `Traefik SSL`\n- Input port : `443`\n- Target port : `443`\n- Device : `bananapim5`\n- Protocol : `TCP`\n\nWe will configure **Traefik** later to **redirect** HTTP requests to HTTPS.\nBut if you prefer you can only open the HTTPS port (if you are going to use Let's encrypt' **HTTP challenge**,\nit's enough for the TLS certificates to be generated, see the warning box a little further below though).\n\n### Allow access through VPN\n\nIf you want some applications to be available from the outside **through VPN**, then simply open the **VPN** port :\n\nGo to your router configuration and add a **port forward rule** for the **UDP** port `51820` :\n\n- Name : `VPN`\n- Input port : `51820`\n- Target port : `51820`\n- Device : `bananapim5`\n- Protocol : `UDP`\n\nOf course if you want the applications to be available **only through VPN**, then only open the VPN port, remove any open HTTP/HTTPS port.\n\n\u003e [!WARNING]\n\u003e Note that if you use Let's Encrypt' **HTTP challenge** to issue and renew **SSL/TLS certificates**, target websites must be reachable from the internet.\n\u003e That mean you will have to open the HTTP(S) port at least when issuing/renewing certificates,\n\u003e you could also keep them open and restrict access to the necessary IP ranges, if your router supports that.\n\u003e If you really don't want to open HTTP(S) ports (better for security), then you will have to configure **DNS challenge** instead of HTTP challenge,\n\u003e if your DNS provider support it.\n\u003e See [HTTP challenge](#http-challenge) and [DNS challenge](#dns-challenge) below when configuring **Traefik**.\n\n## Reverse proxy\n\n\u003cimg src=\"images/logo-traefik.svg\" alt=\"Docker logo\" height=\"148\"/\u003e\n\n**Traefik** is an open source **HTTP reverse proxy** and **load balancer** that can integrate easily with our Docker infrastructure.\nWe will use it to intercept and route every incoming request to the corresponding backend services.\n\nIt will listen to our services and instantly generates the **routes**, so that they are connected to the outside world.\nWe will also use it to automatically generate and renew **SSL/TLS certificates** through **Let's Encrypt**.\n\nHere is an overview of the network flow on our setup :\n\n```mermaid\nflowchart LR\n style INCOMING_REQUEST fill:#205566,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style MYAPP1_CONTAINER fill:#663535,color:#fff\n style MYAPP2_CONTAINER fill:#663535,color:#fff\n style TRAEFIK_ROUTER fill:#806030,color:#fff\n style TRAEFIK_MIDDLEWARE fill:#806030,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665555,color:#fff\n style CONTAINER_ENGINE fill:#664545,color:#fff\n INCOMING_REQUEST((INCOMING\u003cbr/\u003eREQUEST))\n DOCKER_TRAEFIK_PORT443{{433/tcp}}\n DOCKER_TRAEFIK_PORT80{{80/tcp}}\n DOCKER_TRAEFIK_PORT8080{{8080/tcp}}\n DOCKER_MYAPP1_PORT{{exposed port}}\n DOCKER_MYAPP2_PORT{{exposed port}}\n TRAEFIK_ROUTER_MYAPP1(myapp1.example.com)\n TRAEFIK_ROUTER_MYAPP2(myapp2.example.com)\n TRAEFIK_ROUTER_TRAEFIK(traefik.example.com)\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI M5]\n subgraph CONTAINER_ENGINE[DOCKER]\n subgraph MYAPP1_CONTAINER[MYAPP1 CONTAINER]\n DOCKER_MYAPP1_PORT\n end\n subgraph MYAPP2_CONTAINER[MYAPP2 CONTAINER]\n DOCKER_MYAPP2_PORT\n end\n\n subgraph TRAEFIK_CONTAINER[TRAEFIK CONTAINER]\n subgraph TRAEFIK_ROUTER[TRAEFIK HTTP ROUTER]\n TRAEFIK_ROUTER_TRAEFIK\n TRAEFIK_ROUTER_MYAPP1\n TRAEFIK_ROUTER_MYAPP2\n end\n subgraph TRAEFIK_MIDDLEWARE[TRAEFIK MIDDLEWARE]\n REDIRECT(HTTPS redirect)\n IP_WHITELISTING(IP whitelist)\n BASIC_AUTH(Basic auth)\n end\n DOCKER_TRAEFIK_PORT80\n DOCKER_TRAEFIK_PORT443\n DOCKER_TRAEFIK_PORT8080\n end\n\n end\n end\n\n INCOMING_REQUEST --\u003e DOCKER_TRAEFIK_PORT80\n INCOMING_REQUEST --\u003e DOCKER_TRAEFIK_PORT443\n DOCKER_TRAEFIK_PORT80 --\u003e TRAEFIK_ROUTER\n DOCKER_TRAEFIK_PORT443 --\u003e TRAEFIK_ROUTER\n TRAEFIK_ROUTER_TRAEFIK --\u003e REDIRECT\n TRAEFIK_ROUTER_MYAPP1 --\u003e REDIRECT\n TRAEFIK_ROUTER_MYAPP2 --\u003e REDIRECT\n REDIRECT -.-\u003e DOCKER_TRAEFIK_PORT443\n IP_WHITELISTING --\u003e BASIC_AUTH\n IP_WHITELISTING ---\u003e DOCKER_MYAPP2_PORT\n REDIRECT --\u003e IP_WHITELISTING\n REDIRECT ---\u003e DOCKER_MYAPP1_PORT\n BASIC_AUTH --\u003e DOCKER_TRAEFIK_PORT8080\n```\n\nIt handles HTTP to HTTPS redirection, IP whitelisting and basic authentication through custom **middlewares**.\nIn this example `myapp1` is accessible from the internet, `myapp2` is accessible only through VPN,\nand Traefik (dashboard and APIs) is accessible only through VPN after basic authentication.\n\nI've deliberately left out **Sablier** for the moment, to keep things simple, but basically this would simply add a middleware that checks the state of the application,\nin order to temporarily display a waiting page while not ready, refer to [Scale to zero with Sablier](#scale-to-zero-with-sablier) for more information.\n\n### Installation\n\nFirst, create a folder to hold data and configuration :\n\n```bash\nsudo mkdir /opt/apps/traefik\n```\n\nThen copy the files from this project's _traefik_ directory into the _/opt/apps/traefik_ directory :\n\n- _docker-compose.yml_ : The Traefik service definition\n- _traefik.yml_ : The Traefik static configuration\n- _credentials.txt_ : A file that will hold users credentials to access the Traefik dashboard (restricted with **basic authentication**),\n see [Generate basic authentication credentials](#generate-basic-authentication-credentials)\n\nFiles should be ready to use, simply replace the e-mail address (`admin@example.com`) in the _traefik.yaml_ file with your e-mail address.\n\nYou will also need to create the **JSON** file to hold the certificates, see [TLS certificates](#tls-certificates).\n\nAnyway you will find below more details about each file (see [Configuration files details](#configuration-files-details)) and some further configuration.\n\n### Generate basic authentication credentials\n\nAs we configured the Traefik dashboard to be protected with **basic authentication**, allowed users have to be added to the _credentials.txt_ file.\n\nYou can generate a user/password using **htpasswd** :\n\n1. Install the needed package if not present :\n\n ```bash\n sudo apt install apache2-util\n ```\n\n2. Generate the credentials (we use **bcrypt** with a computing time of 10) :\n\n ```bash\n htpasswd -nbBC 10 admin xxxxxxxx\n ```\n\nThen copy the output to the _credentials.txt_ file.\n\n\u003e [!NOTE]\n\u003e Actually as Traefik will be accessible only from local network and through VPN, we don't really need to set up basic authentication,\n\u003e but it's more for demonstration, and it's always better to have 2 layers of security than one.\n\n### TLS certificates\n\n\u003cimg src=\"images/logo-letsencrypt.svg\" alt=\"Let's Encrypt logo\" height=\"64\"/\u003e\n\nTo enable **HTTPS** on our websites, we need to get **TLS certificates** from a **certificate authority**.\nA TLS certificate certifies, in a way, the authenticity of a website (actually it proves that we have the ownership of the public key used for TLS encryption),\npreventing hackers from intercepting any data transmitted between a device and the site.\n\nWe will use **Let's Encrypt**, a nonprofit certificate authority which provide free TLS certificates.\n\n**Let's Encrypt** can automatically generate certificates via Traefik, for that we need to create a `acme.json` file that will hold the generated certificates\n(file is mapped to a volume in the**Compose** file),\nso that the certificates are persisted between container restarts (not generated each time which could raise Let's Encrypt rate limits), we also need to change\nthe permissions so that Traefik can access and edit this file :\n\n```bash\ncd /opt/apps/traefik\ntouch /opt/apps/traefik/acme.json\nchmod 600 /opt/apps/traefik/acme.json\n```\n\n#### HTTP challenge\n\nIf you use **HTTP challenge**, Let's Encrypt will validate that you control the domain names by trying to reach the web server through HTTP or HTTPS.\nSo you must open and port forward ports `80` or `443` for the TLS certificate to be issued correctly.\n\nThe corresponding certificate resolver configuration would be :\n\n```yaml\ntlsChallenge: { }\n```\n\n\u003e [!WARNING]\n\u003e Note that Let’s Encrypt will not let you use this challenge to issue wildcard certificates.\n\n#### DNS challenge\n\nWhen using **DNS challenge**, Let's Encrypt will validate that you control the domain names by querying the DNS system for a TXT record under the target domain name.\nSo you **don't** need to open HTTP or HTTPS port on your router.\n\nFirst, check that your DNS provider is supported by Traefik to automate the DNS verification, a list can be found here : https://doc.traefik.io/traefik/https/acme/.\n\nThen :\n\n1. Create an **access token** / **API key** from your provider interface\n2. Add the necessary **environment variables** required by your provider, to the Traefik service configuration, i.e. :\n ```yaml\n environment:\n MYPROVIDER_ACCESS_TOKEN: \u003caccess_token_here\u003e\n ```\n\nThe corresponding certificate resolver configuration would be :\n\n```yaml\ndnsChallenge:\n provider: \u003cyour_provider_here\u003e\n```\n\n### IP whitelisting\n\nWe will set up **IP whitelisting** so that we can allow only traffic from the local network or from the VPN for some of our services.\nIndeed, even if we do not have defined public subdomains for these services, they can still be reached via the IP address\n(actually in that case Traefik will not route the request, but it is still better to have this additional security).\n\nBasically it involves creating a **Traefik middleware** for defining the IP whitelist and apply it to the needed services.\n\nSo we need to allow 2 **IP ranges** :\n\n- The **local IP range** : IPs assigned to the devices on your local network (computers, mobile devices, ...)\n- The **Traefik Docker bridge network IP range** : IPs assigned by Docker to any container in the Traefik network\n\nFor the Traefik Docker network IP range, you can either take the default assigned one, or assign a static subnet when creating the Traefik network, i.e. :\n\n```yaml\nnetworks:\n traefik-net:\n name: traefik-net\n ipam:\n config:\n - subnet: 172.22.0.0/16\n```\n\nThat way :\n\n- Requests coming from the local network will come with a local address assigned by the router DHCP, and will be **accepted**.\n- Requests coming from the internet through VPN will go through Pi-Hole and will be redirected to Traefik (Pi-hole's local DNS records)\n and thus come with a Traefik Docker network assigned IP address, and will be **accepted**.\n- Requests coming from the internet without VPN will come with a public IP address and will be **rejected** as it will not match any whitelisted address.\n\n### Configuration files details\n\n#### Static configuration file :\n\n:page_facing_up: _traefik.yaml_ :\n\n```yaml\napi:\n dashboard: true\n\nentryPoints:\n web:\n address: ':80'\n\n websecure:\n address: ':443'\n\nproviders:\n docker:\n watch: true\n exposedByDefault: false\n\ncertificatesResolvers:\n default:\n acme:\n email: admin@example.com\n storage: acme.json\n caServer: 'https://acme-v02.api.letsencrypt.org/directory'\n dnsChallenge:\n provider: \u003cyour_provider_here\u003e\n\nlog:\n level: info\n```\n\nThis config file :\n\n- enables the Traefik **dashboard** (UI that provides a detailed overview of the current configuration)\n- defines 2 **entrypoints**, named `web` (for port `80`) and `websecure` (for port `443`) so that we can receive requests on these ports\n- defines a `docker` provider so that we can use **container labels** for retrieving routing configuration. We have configured it to **not** expose containers by default, so\n that containers that do not have a `traefik.enable=true` label are ignored from the resulting routing configuration\n- defines a `default` **certificate resolver** for Let's Encrypt to automatically generate certificates\n- set log level to `info` (you can set it to `debug` when you need more information on what's going on)\n\n#### Service definition :\n\n:page_facing_up: _docker-compose.yml_ :\n\n```yaml\nversion: \"3.7\"\n\nservices:\n\n traefik:\n image: traefik:latest\n container_name: traefik\n ports:\n - \"80:80\"\n - \"443:443\"\n volumes:\n - /var/run/docker.sock:/var/run/docker.sock # So that Traefik can listen to the Docker events\n - ./traefik.yml:/etc/traefik/config.yml # Traefik configuration\n - ./acme.json:/acme.json # For Let's Encrypt certificate storage\n - ./credentials.txt:/credentials.txt:ro # For Traefik dashboard credentials\n networks:\n - traefik-net\n environment:\n MYPROVIDER_ACCESS_TOKEN: \u003caccess_token_here\u003e\n labels:\n - \"traefik.enable=true\"\n\n # Redirect all HTTP requests to HTTPS\n - \"traefik.http.middlewares.httpsonly.redirectscheme.scheme=https\"\n - \"traefik.http.middlewares.httpsonly.redirectscheme.permanent=true\"\n - \"traefik.http.routers.httpsonly.rule=HostRegexp(`{any:.*}`)\"\n - \"traefik.http.routers.httpsonly.middlewares=httpsonly\"\n\n # Configure dashboard with HTTPS\n - \"traefik.http.routers.dashboard.rule=Host(`traefik.example.com`)\"\n - \"traefik.http.routers.dashboard.entrypoints=websecure\"\n - \"traefik.http.routers.dashboard.service=dashboard@internal\"\n - \"traefik.http.routers.dashboard.tls=true\"\n - \"traefik.http.routers.dashboard.tls.certresolver=default\"\n\n # Configure API with HTTPS\n - \"traefik.http.routers.api.rule=Host(`traefik.example.com`) \u0026\u0026 PathPrefix(`/api`)\"\n - \"traefik.http.routers.api.entrypoints=websecure\"\n - \"traefik.http.routers.api.service=api@internal\"\n - \"traefik.http.routers.api.tls=true\"\n - \"traefik.http.routers.api.tls.certresolver=default\"\n\n # IP whitelist for services to be accessible only through VPN and from the local network, have to be applied on each service configuration that need it\n - \"traefik.http.middlewares.vpn-whitelist.ipwhitelist.sourcerange=192.168.0.0/24, 172.18.0.0/16\"\n\n # Secure dashboard/API with authentication\n - \"traefik.http.routers.dashboard.middlewares=auth\"\n - \"traefik.http.routers.api.middlewares=auth\"\n - \"traefik.http.middlewares.auth.basicauth.usersfile=/credentials.txt\"\n\nnetworks:\n\n traefik-net:\n name: traefik-net\n```\n\nThis **Compose** file mainly :\n\n- exposes ports `80` and `443` to receive incoming HTTP/HTTPS requests\n- defines a `traefik-net` **network** (which will have to be shared with the services that will use Traefik)\n- defines an environment variable to hold the DNS provider access token to be able to issue Let's Encrypt certificates through **DNS challenge**\n- defines an HTTP **router** that will match `traefik.example.com` URL on our `websecure` **entrypoint** to point to our service\n- defines `httpsonly` **router** and **middleware** responsible for automatically redirecting HTTP requests to HTTPS\n- configures `dashboard` and `api` routers to use secure HTTPS endpoint with our certificate resolver to generate related Let's Encrypt certificates\n- secures dashboard and API endpoints by defining a `auth` middleware that will handle basic authentication (from _credentials.txt_ file)\n- defines a `vpn-whitelist` **middleware** responsible for whitelisting IPs, so that it can be used by services that will be exposed to the internet to allow only local traffic and\n VPN traffic\n\n\u003e [!CAUTION]\n\u003e The order in which the middlewares are defined in relation to a router is important, they will be applied in the same order as their declaration.\n\n### Run\n\nFinally, run the Compose file :\n\n```bash\nsudo docker-compose -f /opt/apps/traefik/docker-compose.yml up -d\n# You may need to force recreate if you changed a config from an already running configuration\nsudo docker-compose -f /opt/apps/traefik/docker-compose.yml up -d --force-recreate\n```\n\nYou should end-up with a running `traefik` container.\n\nIt should also have generated the needed Let's Encrypt certificates in the _acme.json_ file.\n\nSo you can reach the dashboard at https://traefik.example.com.\n\n\u003cimg src=\"images/screen-traefik.png\" alt=\"Traefik dashboard screenshot\"/\u003e\n\n## VPN and ad-blocking\n\n\u003ctable\u003e\n \u003ctr\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-wireguard.svg\" alt=\"Wireguard logo\" height=\"128\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-pihole.svg\" alt=\"Pi-Hole logo\" height=\"128\"/\u003e\n \u003c/td\u003e\n \u003ctd\u003e\n \u003cimg src=\"images/logo-unbound.svg\" alt=\"Unbound logo\" height=\"128\"/\u003e\n \u003c/td\u003e\n \u003c/tr\u003e\n\u003c/table\u003e\n\nWe will install **WireGuard**, **Pi-hole** and **Unbound** to create a virtual private network (VPN) with ad-blocking and DNS privacy/caching capabilities.\n\n**WireGuard** is a free and open-source modern VPN that utilizes state-of-the-art cryptography to securely encapsulates **IP packets** over **UDP**,\nin order to lower the environment attack surface. As a VPN it establishes a secure connection between a computer and the internet\nby making all the traffic going through an encrypted **tunnel**. The point of self-hosting our own VPN server is to ensure\na **private** and **secure** connection to our services from the internet, without having to trust third-party VPN providers,\nand to keep complete freedom and control over the browsing data.\n\n**Pi-hole** is a network-level ad blocking and internet tracker blocking application.\nIt has the ability to block traditional website advertisements as well as advertisements in unconventional places such as mobile apps ads.\nIt can also be used as a **DNS** server and has a built-in **DHCP** server.\n\n**Unbound** is a validating, recursive, caching **DNS resolver**, that has the ability to contact **DNS authority** servers directly\nin order to validate and cache the queries on your network and serve them to you directly,\nso you don’t have to rely on your ISP or third-party DNS resolvers (like Cloudflare or Google).\n\nWe will also install **WireGuard-UI** which provide a GUI for easier WireGuard configuration and monitoring.\n\nSo the idea is that every client in any network can use the VPN to reach our applications while taking advantage of Pi-Hole and Unbound :\n\n```mermaid\nflowchart TB\n style WINDOWS11 fill:#205566,color:#fff\n style LAPTOP fill:#205566,color:#fff\n style MOBILE fill:#205566,color:#fff\n style MACOS fill:#205566,color:#fff\n style WIREGUARD_SERVER fill:#764545,color:#fff\n style PIHOLE fill:#663535,color:#fff\n style UNBOUND fill:#562525,color:#fff\n style INTERNET fill:#4d683b,color:#fff\n style HOME_NETWORK fill:#263555,color:#fff\n style 5G_NETWORK fill:#263555,color:#fff\n style WORK_NETWORK fill:#263555,color:#fff\n style BANANA_PI fill:#504255,color:#fff\n WINDOWS11(Peer 1 \u003cbr/\u003e Home PC - Windows 11)\n LAPTOP(Peer 2 \u003cbr/\u003e Home laptop - Ubuntu 22)\n MOBILE(Peer 3 \u003cbr/\u003e Phone - Android 14)\n MACOS(Peer 4 \u003cbr/\u003e Work PC - MacOS 13)\n WIREGUARD_SERVER(WireGuard server - Secure VPN)\n PIHOLE(Pi-Hole - Firewall \u0026 ad-blocking)\n UNBOUND(Unbound - Custom DNS resolver)\n INTERNET((Internet))\n\n subgraph HOME_NETWORK[Home network]\n WINDOWS11\n LAPTOP\n end\n\n subgraph 5G_NETWORK[Mobile network]\n MOBILE\n end\n\n subgraph WORK_NETWORK[Work network]\n MACOS\n end\n\n subgraph BANANA_PI[Banana Pi]\n WIREGUARD_SERVER\n PIHOLE\n UNBOUND\n end\n\n WINDOWS11 -- WireGuard tunnel --\u003e WIREGUARD_SERVER\n LAPTOP -- WireGuard tunnel --\u003e WIREGUARD_SERVER\n MOBILE -- WireGuard tunnel --\u003e WIREGUARD_SERVER\n MACOS -- WireGuard tunnel --\u003e WIREGUARD_SERVER\n WIREGUARD_SERVER -- DNS queries --\u003e PIHOLE\n PIHOLE -- Filtered DNS queries --\u003e UNBOUND\n UNBOUND -- DNS resolution --\u003e INTERNET\n```\n\nWe will use a single **Compose** file to set up the 3 services as they are tightly linked.\n\n### Installation\n\nFirst, create a folder to hold data and configuration :\n\n```bash\nsudo mkdir /opt/apps/wireguard\n```\n\nThen from this project's _wireguard_ directory, copy into the _/opt/apps/wireguard_ directory :\n\n- the _.env_ file which holds some environment variables to be used in the Compose file\n- the _docker-compose.yml_ file which contains all the Docker services configuration\n\nFor more details about these files, see [Configuration files details](#configuration-files-details-1).\n\nNow let's take a look at the configuration for each service.\n\n### Configuration\n\n#### WireGuard\n\n\u003cimg src=\"images/logo-wireguard-text.svg\" alt=\"WireGuard logo\" height=\"64\"/\u003e\n\nThe Compose file will run a **WireGuard server**, which need to be configured.\n\nFirst, after WireGuard installation, it is recommended to change the permissions of the _wg0.conf_ file (holding the server configuration) :\n\n```shell\nsudo chmod 600 /opt/apps/wireguard/wireguard/wg0.conf\n```\n\nelse in the logs you will see a warning :\n\n\u003e Warning: `/config/wg_confs/wg0.conf' is world accessible\n\nwhich means that the configuration file permissions are too broad as there’s a private key in there, so it is better to restrict it.\n\n##### Global settings\n\nThen you can do the configuration using WireGuard UI (accessible at https://wireguard-ui.example.com) :\n\nIn **Global Settings** menu :\n\n- set **Endpoint Address** to `wireguard.example.com`, this is the public IP address / hostname of the WireGuard server that every client will connect to\n- set **DNS Servers** to `10.2.0.100` (Pi hole address defined in Docker Compose) instead of `1.1.1.1` (Cloudflare) so that all clients traffic goes through Pi-Hole (and then\n Unbound)\n- adapt the **MTU** (Maximum Transmission Unit) to the right value depending on your network (you will also have to tweak it on each pear configuration).\n I had to set it to `1420`, see [VPN connection speed](#vpn-connection-speed) for more detail about finding best MTU value\n\n\u003e [!CAUTION]\n\u003e Setting a non-optimal value for MTU can lead to slow connection.\n\nIn **WireGuard Server** menu :\n\n- set **Server Interface address** to `10.10.1.1/24` which is the IP range (CIDR) to be used by peers in the tunnel (every peer in the network will be able to get an IP\n between `10.10.1.1` and `10.10.1.254`). You can use another address as you wish.\n\n##### Firewall rules\n\nWe have to set some firewall rules as our WireGuard VPN is running in a Docker container, we need to :\n\n- allow packets to be routed through the WireGuard server, by setting up `FORWARD` rules\n- allow WireGuard clients to access the Internet, by configuring **NAT** (Network Address Translation) rules\n\nSo basically we need to deal with 3 interfaces of our container :\n\n- `eth0@ifxx` : virtual interface that route packets from/to the Traefik Docker bridge network, handling incoming traffic from all peers\n- `eth1@ifxx` : virtual interface that route packets from/to the WireGuard container, for communication within the WireGuard container\n- `wg0` : the WireGuard interface\n\n\u003e [!Note]\n\u003e WireGuard typically requires a network interface for each peer, but as all incoming traffic from the WireGuard peers\n\u003e are arriving at the container using the Traefik bridge network assigned IP address, then only one interface is handling incoming traffic from all WireGuard peers\n\nYou can run the following commands to list network interfaces from the container, which may differ depending on your configuration :\n\nFirst get into the container :\n\n```shell\nsudo docker exec -it wireguard bash\n```\n\nThen run :\n\n```shell\nip link show\n```\n\nYou should get something like :\n\n\u003e ```\n\u003e 1: lo: \u003cLOOPBACK,UP,LOWER_UP\u003e ...\n\u003e 5: wg0: \u003cPOINTOPOINT,NOARP,UP,LOWER_UP\u003e ...\n\u003e 19848: eth1@if19849: \u003cBROADCAST,MULTICAST,UP,LOWER_UP\u003e ...\n\u003e 19850: eth0@if19851: \u003cBROADCAST,MULTICAST,UP,LOWER_UP\u003e ...\n\u003e ```\n\n`ip a` or `ifconfig` will give you the ip address it points to :\n\n\u003e ```\n\u003e eth0 Link encap:Ethernet HWaddr 02:43:AC:2C:00:08\n\u003e inet addr:172.22.0.7 Bcast:172.22.255.255 Mask:255.255.0.0\n\u003e UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1\n\u003e [...]\n\u003e \n\u003e eth1 Link encap:Ethernet HWaddr 02:43:0B:03:00:04\n\u003e inet addr:10.2.0.3 Bcast:10.2.0.255 Mask:255.255.255.0\n\u003e UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1\n\u003e [...]\n\u003e \n\u003e lo Link encap:Local Loopback\n\u003e inet addr:127.0.0.1 Mask:255.0.0.0\n\u003e UP LOOPBACK RUNNING MTU:65536 Metric:1\n\u003e [...]\n\u003e \n\u003e wg0 Link encap:UNSPEC HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00\n\u003e inet addr:10.10.1.1 P-t-P:10.10.1.1 Mask:255.255.255.0\n\u003e UP POINTOPOINT RUNNING NOARP MTU:1450 Metric:1\n\u003e [...]\n\u003e ```\n\nWell, in **WireGuard Server** menu :\n\n- set **Post Up Script** to :\n ```\n iptables -A FORWARD -i %1 -j ACCEPT;\n iptables -A FORWARD -o %1 -j ACCEPT;\n iptables -t nat -A POSTROUTING -o eth+ -j MASQUERADE\n ```\n- set **Post Down Script** to :\n ```\n iptables -D FORWARD -i %1 -j ACCEPT;\n iptables -D FORWARD -o %1 -j ACCEPT;\n iptables -t nat -D POSTROUTING -o eth+ -j MASQUERADE\n ```\n\n**Post Up** and **Post Down** defines steps to be run after the interface is turned on or off, respectively.\nIn this case, **iptables** is used to set IP rules.\nThe rules will then be cleared once the tunnel is down.\n\n\u003e [!Note]\n\u003e I used the old deprecated **iptables** to set firewall rules, but you may better use **nftables** which is the successor to iptables\n\n`%1` is a placeholder for the network interface connected to the WireGuard container, so here `wg0`.\n`eth+` is a pattern used in iptables to match network interfaces that start with the prefix `eth`, so it matches our 2 virtual interfaces.\n\nThe first 2 rules allow packets to be forwarded between interfaces, for traffic originating from the WireGuard interface `wg0` (rule 1), and heading out of `wg0` (rule 2).\nThese two rules allow forwarding so every traffic going in or out of the WireGuard interface can be forwarded (routed).\nThe last rule translates incoming IPs to the IP on every `eth` interface, so basically **NAT**.\n\nYou can see that iptables are applied by running :\n\n```shell\niptable -L\n```\n\nResult :\n\u003e ```\n\u003e Chain INPUT (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e \n\u003e Chain FORWARD (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e ACCEPT all -- anywhere anywhere\n\u003e ACCEPT all -- anywhere anywhere\n\u003e \n\u003e Chain OUTPUT (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e ```\n\nAnd for the NAT table :\n\n```shell\niptable -t nat -L\n```\n\nResult :\n\u003e ```\n\u003e Chain PREROUTING (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e\n\u003e Chain INPUT (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e\n\u003e Chain OUTPUT (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e\n\u003e Chain POSTROUTING (policy ACCEPT)\n\u003e target prot opt source destination\n\u003e MASQUERADE all -- anywhere anywhere\n\u003e ```\n\n##### Clients\n\nIn **WireGuard Clients** settings, create a new client :\n\n- name : `desktop-home` (for example)\n- e-mail : `your.email@example.com`\n\nIt should propose IP allocation of `10.10.1.2/32` for first client, then `10.10.1.3/32`, and so on as we set server interface address to `10.10.1.1/24`.\n\nThen you can configure **Allowed IPs**, which is used to tell the server the IP addresses to which it will allow client to connect (it will set routes for the peer on your host).\nThis should be set to all the addresses you want to have access inside the tunnel.\n\nBy default, it is set to `0.0.0.0/0`, which represents all IPv4 addresses (full tunneling), and will block all non-tunneled traffic\n(WireGuard client will automatically check a \"Block untunneled traffic (kill-switch)\" checkbox in the UI).\n\nKeeping `0.0.0.0/0` caused problems in my case on my main Windows machine (local server and hosts not reachable).\nIndeed, this blocks access to local network resources, as all traffic, including local network traffic,\nis routed through the WireGuard tunnel, thus preventing direct communication with devices on the local network.\n\nSo I had to change it to `0.0.0.0/1, 128.0.0.0/1` (or just uncheck the checkbox), to make it work as expected.\nUsing `/1` instead of `/0` ensure that it takes precedence over the default `/0` route.\n\nTo understand what's going on, you can check the routes on Windows by running the `route print` command to see what is modified when you change allowed IPs :\n\nWhen using `0.0.0.0/0` :\n\n```cmd\n===========================================================================\nNetwork Destination \t Netmask \t Gateway \t Interface Metric\n 0.0.0.0 0.0.0.0 On-link 10.10.1.2 0\n 192.168.0.0 255.255.255.0 On-link 192.168.0.11 281\n ... ... ... ... ...\n===========================================================================\n```\n\nWhen using `0.0.0.0/1, 128.0.0.0/1` :\n\n```cmd\n===========================================================================\nNetwork Destination \t Netmask \t Gateway \t Interface Metric\n 0.0.0.0 128.0.0.0 On-link 10.10.1.2 5\n 127.255.255.255 255.255.255.255 On-link 10.10.1.2 261\n 128.0.0.0 128.0.0.0 On-link 10.10.1.2 5\n 192.168.0.0 255.255.255.0 On-link 192.168.0.11 281\n ... ... ... ... ...\n===========================================================================\n```\n\nThe first config covers the entire IP range with one route while the second config splits the IP range into two halves,\neach covered by a separate route.\n\nAs I understand this second setup is a form of split tunneling that routes most traffic through the VPN\nwhile still allowing direct access to the local network resources.\nIndeed, the route `192.168.0.11` (desktop PC's local IP address) is more specific than the `0.0.0.0/1 ` and `128.0.0.0/1 ` routes, so it takes precedence.\nThat way traffic to `192.168.0.x` addresses will match these more specific routes and be sent directly to the local network interface (`192.168.0.11`), bypassing the VPN.\nIt's more flexible than the `0.0.0.0/0` configuration, which would route all traffic (including local) through the VPN.\n\nOn other clients this is not necessarily required, it depends on the routing table.\n\nAnyway, to run a VPN client :\n\n1. Export config file for your client\n2. Install WireGuard client on your client machine\n3. Load config file from client\n\nDo this for each client on every device you need.\n\n\u003cimg src=\"images/screen-wireguard-ui.png\" alt=\"WireGuard-UI screenshot\"/\u003e\n\n#### Pi-hole\n\n\u003cimg src=\"images/logo-pihole.svg\" alt=\"Pi-Hole logo\" height=\"128\"/\u003e\n\nThe Compose file will run a **Pi-Hole** instance which need to be configured.\n\nFirst, we need to change **interface settings** to allow the traffic from other interfaces (especially for our VPN).\nBy default, it allows only queries from local devices (from the same network as the Pi-Hole's network).\n\nSo, reach Pi-Hole at https://pihole.example.com and go to _Settings -\u003e Interface settings_ and choose _\"Permit all origins\"_ instead of default _\"Allow only local requests\"_,\nso that the traffic from outside the Docker bridge network can be seen (indeed, \"local\" for Pi-Hole is the Docker bridge network,\nand thus it would allow only queries from inside that network).\n\nThen we need to add **local DNS records** so that the domain names can be resolved from VPN or local network (remember we have routed all the traffic through Pi-Hole).\nWe simply need to associate domain names with the internal IP address of the Banana Pi, so they can be handled by the reverse proxy.\n\nGo to _local DNS -\u003e DNS records_ and add a **DNS record entry** for every subdomain that should be available through VPN :\n\n```\nackee.example.com 192.168.0.17\ndashboard.example.com 192.168.0.17\ndashdot.example.com 192.168.0.17\nkuma.example.com 192.168.0.17\nphpmyadmin.example.com 192.168.0.17\npihole.example.com 192.168.0.17\nportainer.example.com 192.168.0.17\ntraefik.example.com 192.168.0.17\nwireguard-ui.example.com 192.168.0.17\n```\n\nNo need to add domains that are reachable from the internet as they will be reachable directly over HTTPS without going through our Pi-Hole.\n\nYou can also configure rate limiting (default to **1000 queries per minute**), domain whitelisting, DNS settings, etc. but I will not go through all Pi-Hole configuration, the\ndefault should work just fine.\n\nIf it is working you should be able to see activity in the dashboard.\n\n\u003cimg src=\"images/screen-pihole.png\" alt=\"Pi-hole screenshot\"/\u003e\n\n#### Unbound\n\n\u003cimg src=\"images/logo-unbound.svg\" alt=\"Unbound logo\" height=\"128\"/\u003e\n\nThe first time you will run **Unbound**, it may fail because a few files included in the default configuration will be missing (at least in the image version I'm using),\nindeed the following files are included in the default _unbound.conf_ file (which should have been created correctly in _/etc/unbound/unbound.conf_) :\n\n- _/opt/unbound/etc/unbound/a-records.conf_\n- _/opt/unbound/etc/unbound/srv-records.conf_\n- _/opt/unbound/etc/unbound/forward-records.conf_\n\nYou could manually create these files (you can find default ones from the Unbound **GitHub** repository),\nand then mount them into the Unbound container, before running again the Compose file.\n\nBut that way it would run Unbound in **forwarder** mode, meaning that the DNS server will forward all the queries to **Cloudflare**.\nThis was my first try and a **DNS leak test** confirmed that it uses Cloudflare, indeed the default _forward-records.conf_ file includes the following forwarding rules :\n\n```\nforward-addr: 1.1.1.1@853#cloudflare-dns.com\nforward-addr: 1.0.0.1@853#cloudflare-dns.com\n```\n\nSo, if you want to run Unbound **without forwarding**, just remove the line that includes the _forward-records.conf_ file in the _unbound.conf_ file.\nOr don't create any of the 3 above files at all (and do not bind them in the container), and remove the includes from the _unbound.conf_ file.\n\nA DNS leak test should now show your IP address as DNS server.\n\n\u003e [!IMPORTANT]\n\u003e If you use the default _forward-records.conf_ file, Unbound will run in **forwarder** mode, meaning that it will forward all queries to **Cloudflare**.\n\u003e To remove the default forwarding to Cloudflare and make your unbound container a recursive-only server,\n\u003e edit the _unbound.conf_ file and remove include of the _forward-records.conf_ file.\n\nFinally, if you want to activate **logging** for debugging purposes, edit the _/etc/unbound/unbound.conf_ configuration file :\n\n```\nverbosity: 1\nlog-queries: yes\n```\n\nBut it's not recommended to increase verbosity for daily use, as Unbound logs a lot.\n\n### Configuration files details\n\n#### Environment variables\n\n:page_facing_up: _.env_ :\n\n```shell\nWIREGUARD_UI_USERNAME=\u003cusername\u003e\nWIREGUARD_UI_PASSWORD=\u003cpassword\u003e\nPIHOLE_PASSWORD=\u003cpassword\u003e\n```\n\nIt simply defines environment variables to be used in the Docker Compose file.\n\n#### Services definition\n\n:page_facing_up: _docker-compose.yaml_ :\n\n```yaml\nversion: \"3.7\"\n\nnetworks:\n\n wireguard_net:\n name: wireguard_net\n ipam:\n driver: default\n config:\n - subnet: 10.2.0.0/24\n\n traefik-net:\n name: traefik-net\n external: true\n\nservices:\n\n unbound:\n image: \"mvance/unbound-rpi:latest\"\n container_name: unbound\n restart: unless-stopped\n hostname: \"unbound\"\n volumes:\n - \"./unbound:/opt/unbound/etc/unbound/\"\n networks:\n wireguard_net:\n ipv4_address: 10.2.0.200\n\n wireguard:\n depends_on: [ unbound, pihole ]\n image: linuxserver/wireguard:latest\n container_name: wireguard\n cap_add:\n - NET_ADMIN\n volumes:\n - ./wireguard:/config\n ports:\n - \"51820:51820/udp\"\n restart: unless-stopped\n environment:\n - PUID=1000\n - PGID=1000\n - TZ=Europe/Zurich\n sysctls:\n - net.ipv4.conf.all.src_valid_mark=1\n networks:\n wireguard_net:\n ipv4_address: 10.2.0.3\n traefik-net:\n labels:\n - \"traefik.enable=true\"\n - \"traefik.http.routers.wireguard-ui.rule=Host(`wireguard-ui.example.com`)\"\n - \"traefik.http.routers.wireguard-ui.entrypoints=websecure\"\n - \"traefik.http.routers.wireguard-ui.tls.certresolver=default\"\n - \"traefik.http.routers.wireguard-ui.middlewares=vpn-whitelist\"\n - \"traefik.http.services.wireguard-ui.loadbalancer.server.port=5000\"\n - \"traefik.docker.network=traefik-net\"\n\n wireguard-ui:\n image: ngoduykhanh/wireguard-ui:latest\n container_name: wireguard-ui\n depends_on: [ unbound, wireguard ]\n cap_add:\n - NET_ADMIN\n # use the network of the 'wireguard' service, this enables to show active clients in the status page\n network_mode: service:wireguard\n env_file: ./.env\n environment:\n - SENDGRID_API_KEY\n - EMAIL_FROM_ADDRESS\n - EMAIL_FROM_NAME\n - SESSION_SECRET\n - WGUI_USERNAME=$WIREGUARD_UI_USERNAME\n - WGUI_PASSWORD=$WIREGUARD_UI_PASSWORD\n - WG_CONF_TEMPLATE\n - WGUI_MANAGE_START=true\n - WGUI_MANAGE_RESTART=true\n logging:\n driver: json-file\n options:\n max-size: 50m\n volumes:\n - ./wireguard-ui-db:/app/db\n - ./wireguard:/etc/wireguard\n\n pihole:\n depends_on: [ unbound ]\n container_name: pihole\n image: pihole/pihole:latest\n restart: unless-stopped\n hostname: pihole\n env_file: ./.env\n ports:\n - \"53:53/tcp\"\n - \"53:53/udp\"\n labels:\n - \"traefik.enable=true\"\n - \"traefik.http.routers.pihole.rule=Host(`pihole.example.com`)\"\n - \"traefik.http.routers.pihole.entrypoints=websecure\"\n - \"traefik.http.routers.pihole.tls.certresolver=default\"\n - \"traefik.http.routers.pihole.middlewares=vpn-whitelist\"\n - \"traefik.http.services.pihole.loadbalancer.server.port=80\"\n - \"traefik.docker.network=traefik-net\"\n dns:\n - 127.0.0.1\n - 10.2.0.200 # Unbound IP\n environment:\n TZ: \"Europe/Zurich\"\n WEBPASSWORD: $PIHOLE_PASSWORD\n ServerIP: 10.2.0.100 # Internal IP of pi-hole\n DNS1: 10.2.0.200 # Unbound IP\n DNS2: 10.2.0.200 # If we don't specify two, it will auto pick google.\n volumes:\n - \"./etc-pihole/:/etc/pihole/\"\n - \"./etc-dnsmasq.d/:/etc/dnsmasq.d/\"\n # Recommended but not required (DHCP needs NET_ADMIN)\n cap_add:\n - NET_ADMIN\n networks:\n wireguard_net:\n ipv4_address: 10.2.0.100\n traefik-net:\n```\n\nThis **Compose** file roughly :\n\n- defines a `wireguard_net` **network** to hold our 4 WireGuard-related services, with the assigned subnet address `10.2.0.0/24` (**CIDR** notation)\n- reference the `traefik-net` Traefik network so that services can use it and be discoverable by Traefik\n- defines our 4 services (WireGuard, WireGuard UI, Pi-Hole, Unbound) :\n - `unbound` service :\n - defines a **volume** that binds the configuration folder to a local folder, in case we want to change default configuration\n - assigns the **static IP address** `10.2.0.200` for the container inside the WireGuard network\n - `wireguard` service :\n - defines a **volume** to bind configuration file\n - adds network capability `NET_ADMIN` to grant the container the ability to perform various network-related tasks\n (like configuring network interfaces or changing routing tables) required be the service\n - enables the `net.ipv4.conf.all.src_valid_mark` sysctl setting to activate source address validation, which helps in preventing IP spoofing attacks\n - uses Traefik **labels** to :\n - create a **service** which will point to our container application running on port `5000`\n - create an HTTP **router** that will match `wireguard-ui.example.com` URL on our `websecure` **entrypoint** to point to our service\n - assign the `vpn-whitelist` **middleware** so that the traffic will be restricted to allowed IPs only (application reachable only from local network or through VPN)\n - add a **TLS** configuration that will use our `default` **certificates resolver**, so it can generate Let's encrypt certificates\n - assigns `the` **static IP address** `10.2.0.3` for the container inside the WireGuard network\n - `wireguard-ui` service :\n - add network capability `NET_ADMIN` to grant the container the ability to perform various network-related tasks\n (like configuring network interfaces or changing routing tables) required be the service\n - uses the network of the `wireguard` service\n - references the _.env_ file containing some defined environment variables values\n - defines JSON file logging with a max size of 50 MB\n - `pihole` service :\n - defines **volumes** to bind configuration files\n - references the _.env_ file containing some defined environment variables values\n - uses Traefik **labels** to :\n - create a **service** which will point to our container application running on port `80`\n - create an HTTP **router** that will match `pihole.example.com` URL on our `websecure` **entrypoint** to point to our service\n - assign the `vpn-whitelist` **middleware** so that the traffic will be restricted to allowed IPs only (application reachable only from local network or through VPN)\n - add a **TLS** configuration that will use our `default` **certificates resolver**, so it can generate Let's encrypt certificates\n - sets DNS to point to Unbound\n - adds network capability `NET_ADMIN` to grant the container the ability to perform various network-related tasks\n (like configuring network interfaces or changing routing tables) required be the service\n - assigns the **static IP address** `10.2.0.100` for the container inside the WireGuard network\n\n### Run\n\nSimply run the Compose file :\n\n```bash\nsudo docker-compose -f /opt/apps/wireguard/docker-compose.yml up -d\n```\n\nYou should end-up with **4** running containers :\n\n- `wireguard`\n- `wireguard-ui`\n- `pihole`\n- `unbound`\n\nIt should also have generated the needed Let's Encrypt certificates in the _acme.json_ file.\n\nUnbound is not exposed, but you can reach other services :\n\n- WireGuard server at https://wireguard.example.com\n- WireGuard GUI at https://wireguard-ui.example.com\n- Pi-Hole at https://pihole.example.com\n\n## Test the network\n\n### DNS resolution\n\nEach service should be resolvable through its **subdomain name**.\n\nWhen a user enters the URL in the browser, the browser need to know the IP address corresponding to the domain name, so it can send the queries to. For that it :\n\n1. Checks the **browser cache** (most browsers cache DNS data by default), and use the address corresponding to the provided name if found\n2. Checks the **OS cache** and return the address to the browser if found\n3. Checks the **local host table file** (usually _/etc/hosts_ on Linux/Mac systems and _C:\\Windows\\System32\\Drivers\\etc\\hosts_ on Windows)\n to see if an entry matches the specified name, if so, it will directly return it to the browser\n4. Invokes the **local resolver**, on Windows, it is defined at the network adapter level, usually\n _Control Panel \u003e Network and Internet \u003e Network Connections_ then in the advanced properties of the desired connection (Higher Priority Connection).\n On Linux/Mac it is usually _/etc/resovl.conf_. In my Windows system it is automatically configured to point to my router local IP Address.\n The resolver checks its cache to see if it already has the address for this name. If it does, it returns it immediately to the browser\n5. Checks the **router cache** and return the address to the browser if found\n6. Checks the **ISP cache** and return the address to the browser if found\n7. Checks the **ISP resolving name server** which will call the **root DNS servers** (root server \u003c--\u003e TLD server \u003c--\u003e Authoritative Name Server)\n to find the IP address from the DNS server responsible for the domain name\n\nIn our case every request from the **local network** is forwarded to **Pi-hole**, so the IP resolving will always go through **Pi-Hole** and **Unbound**.\nSee [Network flow](#network-flow) later below for a more graphical representation of the network flow.\n\nYou can first test that each service is resolvable using `nslookup` command, i.e. :\n\n```cmd\nC:\\Users\\Yann39\u003enslookup myapp.example.com\nServer : pi.hole\nAddress: 10.2.0.100\n\nName : myapp.example.com\nAddress: 192.168.0.17\n```\n\nThen you can look for DNS leak using any online checker, to determine which DNS servers the browser is using to resolve domain names,\nit should end up showing your **public IP address**, not Cloudflare or Google, etc. as we use **Unbound** (see [Unbound](#unbound) for configuration).\n\nYou could also use tools like **Wireshark** to look closely at DNS resolution or to confirm that the traffic is effectively going through the VPN when connected\n(in that case the \"Protocol\" column should be `WireGuard` for all queries). I will not go through a Wireshark tutorial, but it is a very\nuseful and interesting tool for viewing what going on in your network.\n\n### Reachability\n\nTo verify that the network is set up correctly, we can simply try to access some services and see if we can reach them or not,\nfrom different device and connection type.\n\n\u003e [!NOTE]\n\u003e I simply temporarily added a `CNAME` record in my domain name registrar for the services to be checked, to point to my DDNS for testing the IP whitelisting,\n\u003e Traefik will not route request if you try to access a service via the public IP address.\n\nFor example if we try to access a service that must be accessible only through VPN (and local network), here are the results :\n\n| Device | Connection | VPN status | Public IP | Remote address (request header) | Traefik | Response |\n|--------|------------|-------------------|---------------|---------------------------------|---------------|---------------------------|\n| PC | cable | :red_circle: off | 144.12.117.3 | 192.168.0.17 | 192.168.0.11 | :heavy_check_mark: 200 OK |\n| PC | cable | :green_circle: on | 144.12.117.3 | 192.168.0.17 | 192.168.0.11 | :heavy_check_mark: 200 OK |\n| Mobile | wifi | :red_circle: off | 144.12.117.3 | 192.168.0.17 | 192.168.0.12 | :heavy_check_mark: 200 OK |\n| Mobile | wifi | :green_circle: on | 144.12.117.3 | 192.168.0.17 | 172.22.0.1 | :heavy_check_mark: 200 OK |\n| Mobile | 4G | :red_circle: off | 81.165.84.189 | 144.12.117.3 | 81.165.84.189 | :x: 403 Forbidden |\n| Mobile | 4G | :green_circle: on | 144.12.117.3 | 192.168.0.17 | 172.22.0.1 | :heavy_check_mark: 200 OK |\n\n- `192.168.0.17` is the Banana Pi's private IP address\n- `144.12.117.3` is the router's public IP address\n- `192.168.0.11` is the desktop PC's local IP address\n- `192.168.0.12` is the mobile phone's local IP address\n- `172.22.0.1` is the Traefik Bridge network IP address\n- `81.165.84.189` is the public IP address on the mobile 4G network\n\nThese are expected results, we can see that the service is reachable from the local network and from anywhere when using the VPN,\nand it is not accessible outside the local network if we don't use the VPN.\n\nWe can also confirm this by looking at the **Traefik logs** (you have to set `level` to `debug` in _traefik.yml_ file to see the debug logs)\nwhich shows that the `vpn-whitelist` **middleware** blocks any IP address that is not whitelisted :\n\n\u003e ```\n\u003e level=debug msg=\"Authentication succeeded\" middlewareType=BasicAuth middlewareName=auth@docker\n\u003e level=debug msg=\"Accepting IP 192.168.0.17\" middlewareName=vpn-whitelist@docker middlewareType=IPWhiteLister\n\u003e level=debug msg=\"Accepting IP 172.22.0.1\" middlewareName=vpn-whitelist@docker middlewareType=IPWhiteLister\n\u003e level=debug msg=\"Rejecting IP 81.165.84.189: \\\"81.165.84.189\\\" matched none of the trusted IPs\" middlewareName=vpn-whitelist@docker middlewareType=IPWhiteLister\n\u003e ```\n\n### VPN connection speed\n\nTo verify that the VPN is not killing the connection speed,\nyou can first use an online **speed test**, this will confirm whether the connection speed is close to normal.\n\nIn my case I observed an abnormally slow connection (**~22 MB/s** download and upload speed, even though I have a gigabit connection whose speed reaches **700+ MB/s** without VPN).\n\n![Ookla test with MTU 1450](images/screen-ookla-test-mtu-1450.png)\n\nThat was because of the WireGuard **MTU** (**Maximum Transmission Unit**) value,\nwhich need to be slightly adjusted.\n\n#### Configure MTU\n\nBy default, WireGuard sets an MTU value of `1450` bytes, which may not be optimal for your connection.\n\nMost of **Ethernet** connections have an MTU of `1500`.\nDocker also sets the MTU of a **bridge** network to `1500` by default.\nYou can confirm this with `ip link | grep mtu`.\n\n```\n...\n5: docker0: \u003cNO-CARRIER,BROADCAST,MULTICAST,UP\u003e mtu 1500 qdisc noqueue state DOWN mode DEFAULT group default\n...\n```\n\nYou can also observe any packet fragmentation on your network by running the `ping` command with the right parameters :\n\n```console\nping www.google.com -f -l 1472\nping www.google.com -f -l 1473\n```\n\nIf the MTU is too high, it will tell you that the packet needs to be fragmented (packets larger than\nthe connection’s MTU size cannot be transmitted and will be fragmented into smaller packets), else ping will answer normally.\n\n`1472` will work and `1473` will warn about fragmented packets,\nthis is because the **IPv4 header** is `20` bytes and the **ICMP header** is `8` bytes (so `1472 + 20 + 8 = 1500`).\n\nBut when going through WireGuard, it also sets additional bytes, which will result in a `60` bytes header, exceeding the value of `1500` (`1450 + 60 = 1510`).\n\nThe worst case (**IPv6**, which has a `40` bytes header compared to `20` bytes of IPv4) ends up being for WireGuard `1500 - 80` = `1420`.\nHowever, if you know that you're going to be using IPv4 exclusively, then you could go with `1440`.\n\nSo just set that value as the MTU for the WireGuard server and peer.\n\nAnother solution, instead of having to set the MTU on each client,\nwould be to clamp the **MSS** on TCP packets to a smaller value on behalf of the clients (i.e. through **iptables**) :\n\n```bash\niptables -t filter -A FORWARD -o wg0 -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1360\niptables -t mangle -A POSTROUTING -o eth0 -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1360\n```\n\n**TCP MSS** is the maximum amount of data in bytes that a host is willing to accept in a single TCP segment.\n\nWhen the TCP traffic will go through the VPN tunnel, additional `60` bytes headers will be added to the original packet (to keep it secure),\nthus using `1360` will prevent the size of the encapsulated packet to go beyond the MTU of the VPN interface (`1420`).\n\n\n#### Measure speed with iPerf\n\nThen you can test the connection speed between the WireGuard server and the peer, using the **iPerf** utility.\n\n1. Enter the WireGuard container :\n\n ```bash\n docker exec -it wireguard bash\n ```\n\n2. Install iPerf (use `apk` as this is an Alpine Linux) :\n\n ```bash \n apk add iperf\n ```\n\n3. Install iPerf on the client machine (**Windows** in my case, so I just downloaded and extracted the _.exe_ file).\n\n4. Run iPerf in **server mode** on the WireGuard server :\n\n ```bash \n iperf --server\n ```\n\n5. Run iPerf in **client mode** on the client machine to execute the test :\n\n ```bash \n iperf --client 10.2.0.3 --time 5 --reverse\n ```\n - `10.2.0.3` is our WireGuard server static IP address\n - `--time 5` runs the test for 5 seconds\n - `--reverse` runs a download test (omit it to test upload)\n\n\u003e [!WARNING]\n\u003e iperf 2 and iperf 3 are not compatible, so make sure to install the same major version on both side,\n\u003e else you may get `iperf3: error - unable to connect to server: Connection refused`\n\nSo here it the output with the default `1450` MTU :\n\n```console\n------------------------------------------------------------\nClient connecting to 10.2.0.3, TCP port 5001\nTCP window size: 208 KByte (default)\n------------------------------------------------------------\n[ 3] local 10.10.1.2 port 52846 connected with 10.2.0.3 port 5001\n[ ID] Interval Transfer Bandwidth\n[ 3] 0.0- 5.3 sec 14.9 MBytes 23.7 Mbits/sec\n```\n\nWith `1420` MTU :\n\n```console\n------------------------------------------------------------\nClient connecting to 10.2.0.3, TCP port 5001\nTCP window size: 208 KByte (default)\n------------------------------------------------------------\n[ 3] local 10.10.1.2 port 51999 connected with 10.2.0.3 port 5001\n[ ID] Interval Transfer Bandwidth\n[ 3] 0.0- 5.0 sec 225 MBytes 378 Mbits/sec\n```\n\nAnd even better with `1400` MTU :\n\n```console\n------------------------------------------------------------\nClient connecting to 10.2.0.3, TCP port 5001\nTCP window size: 208 KByte (default)\n------------------------------------------------------------\n[ 3] local 10.10.1.2 port 53401 connected with 10.2.0.3 port 5001\n[ ID] Interval Transfer Bandwidth\n[ 3] 0.0- 5.0 sec 247 MBytes 414 Mbits/sec\n```\n\nOther values give roughly the same results.\n\nYou can also run multiple tests in parallel, with the `-P` argument :\n\n```bash\niperf --client 10.2.0.3 --time 5 --reverse -P 3\n```\n\n```console\n------------------------------------------------------------\nClient connecting to 10.2.0.3, TCP port 5001\nTCP window size: 208 KByte (default)\n------------------------------------------------------------\n[ 3] local 10.10.1.2 port 60001 connected with 10.2.0.3 port 5001\n[ 5] local 10.10.1.2 port 60003 connected with 10.2.0.3 port 5001\n[ 4] local 10.10.1.2 port 60002 connected with 10.2.0.3 port 5001\n[ ID] Interval Transfer Bandwidth\n[ 3] 0.0- 5.0 sec 89.8 MBytes 150 Mbits/sec\n[ 4] 0.0- 5.0 sec 90.1 MBytes 151 Mbits/sec\n[ 5] 0.0- 5.0 sec 78.2 MBytes 131 Mbits/sec\n[SUM] 0.0- 5.0 sec 258 MBytes 432 Mbits/sec\n```\n\nAnd that way we can see that the **CPU load** on the Banana Pi reaches 100% and can limit the bandwidth :\n\n![CPU load during iPerf test](images/screen-cpu-load-iperf.png \"CPU load during iPerf test\")\n\nAnyway, we improved a lot ! A new online test confirms it :\n\n![Ookla test with MTU 1450](images/screen-ookla-test-mtu-1400.png \"Ookla test with MTU 1450\")\n\n\u003e [!NOTE]\n\u003e You can try other value to see what fits best in your network.\n\u003e There are other parameters than can influence the connection speed (CPU load, distance, etc.), but I stopped investigation here as it's performing well enough for my use.\n\n## Network flow\n\nFor the following examples, we will consider that the **user** enters http://myapp.example.com in the **browser** for the first time (no **DNS record** found in cache).\n\n### Without VPN\n\nHere is what happen when you try to reach a service which is **open to the internet**, without using any VPN,\nfrom your local network holding your homelab (on the left), or from any other location (on the right) :\n\n\u003ctable width=\"100%\"\u003e\n\u003ctr\u003e\n \u003cth\u003eFrom local network\u003c/th\u003e\n \u003cth\u003eFrom outside local network\u003c/th\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n\u003ctd width=\"480px\"\u003e\n\n```mermaid\nflowchart TB\n style HOSTING_PROVIDER fill:#4d683b,color:#fff\n style DDNS_PROVIDER fill:#69587b,color:#fff\n style INTERNET_SERVICE_PROVIDER fill:#205566,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n style CONTAINER_ENGINE fill:#664343,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style PIHOLE_CONTAINER fill:#663535,color:#fff\n style UNBOUND_CONTAINER fill:#663535,color:#fff\n style MYAPP_CONTAINER fill:#663535,color:#fff\n style TRAEFIK_ROUTER fill:#806030,color:#fff\n style TRAEFIK_MIDDLEWARE fill:#806030,color:#fff\n DOMAIN(example.com)\n SUBDOMAIN_MYAPP(myapp.example.com)\n DDNS(myddns.ddns.net)\n ROUTER_PUBLIC_IP[public IP]\n ROUTER_PORT80{{80/tcp}}\n ROUTER_PORT443{{443/tcp}}\n ROUTER_DNS[DNS]\n DOCKER_PIHOLE_PORT53{{53/udp}}\n DOCKER_TRAEFIK_PORT443{{433/tcp}}\n DOCKER_TRAEFIK_PORT80{{80/tcp}}\n DOCKER_MYAPP_PORT{{port/tcp}}\n DOCKER_UNBOUND_PORT53{{53/udp}}\n TRAEFIK_ROUTER_MYAPP(myapp.example.com)\n TRAEFIK_MIDDLEWARE_REDIRECT(HTTPS redirect)\n ROOT_DNS_SERVERS[Root DNS servers]\n\n subgraph HOSTING_PROVIDER[DOMAIN NAME REGISTRAR]\n DOMAIN\n SUBDOMAIN_MYAPP\n end\n\n subgraph DDNS_PROVIDER[DYNAMIC DNS PROVIDER]\n DDNS\n end\n\n subgraph INTERNET_SERVICE_PROVIDER[INTERNET SERVICE PROVIDER]\n ROUTER_PUBLIC_IP\n ROUTER_PORT80\n ROUTER_PORT443\n ROUTER_DNS\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI M5]\n subgraph CONTAINER_ENGINE[DOCKER]\n subgraph MYAPP_CONTAINER[MYAPP CONTAINER]\n DOCKER_MYAPP_PORT\n end\n\n subgraph UNBOUND_CONTAINER[UNBOUND CONTAINER]\n DOCKER_UNBOUND_PORT53\n end\n\n subgraph PIHOLE_CONTAINER[PIHOLE CONTAINER]\n DOCKER_PIHOLE_PORT53\n end\n\n subgraph TRAEFIK_CONTAINER[TRAEFIK CONTAINER]\n DOCKER_TRAEFIK_PORT443\n DOCKER_TRAEFIK_PORT80\n\n subgraph TRAEFIK_ROUTER[TRAEFIK HTTP ROUTER]\n TRAEFIK_ROUTER_MYAPP\n end\n\n subgraph TRAEFIK_MIDDLEWARE[TRAEFIK MIDDLEWARE]\n TRAEFIK_MIDDLEWARE_REDIRECT\n end\n end\n\n end\n\n end\n\n CLIENT((client)) ---\u003e|\" http‎://myapp.example.com \"| BROWSER\n BROWSER((browser)) --\u003e|HTTP| ROUTER_PUBLIC_IP\n DOMAIN \u003c--\u003e|subdomain| SUBDOMAIN_MYAPP\n SUBDOMAIN_MYAPP \u003c--\u003e|CNAME| DDNS\n DDNS \u003c--\u003e|DynDNS| ROUTER_PUBLIC_IP\n ROUTER_PUBLIC_IP --\u003e ROUTER_PORT80\n ROUTER_PUBLIC_IP --\u003e ROUTER_PORT443\n ROUTER_PORT443 --\u003e|port forward| DOCKER_TRAEFIK_PORT443\n ROUTER_PORT80 --\u003e|port forward| DOCKER_TRAEFIK_PORT80\n DOCKER_TRAEFIK_PORT443 --\u003e TRAEFIK_ROUTER\n DOCKER_TRAEFIK_PORT80 --\u003e TRAEFIK_ROUTER\n TRAEFIK_ROUTER_MYAPP --\u003e TRAEFIK_MIDDLEWARE_REDIRECT\n TRAEFIK_MIDDLEWARE_REDIRECT --\u003e DOCKER_TRAEFIK_PORT443\n TRAEFIK_MIDDLEWARE_REDIRECT --\u003e DOCKER_MYAPP_PORT\n BROWSER((browser)) \u003c--\u003e LOCAL_DNS_RESOLVER[/local resolver\\]\n LOCAL_DNS_RESOLVER \u003c---\u003e|router local IP address| ROUTER_DNS\n ROUTER_DNS \u003c--\u003e|Banana Pi M5 static IP| DOCKER_PIHOLE_PORT53\n DOCKER_PIHOLE_PORT53 \u003c--\u003e|DNS| DOCKER_UNBOUND_PORT53\n UNBOUND_CONTAINER \u003c-----\u003e ROOT_DNS_SERVERS\n linkStyle 0 stroke-width: 4px, stroke: red\n linkStyle 1 stroke-width: 4px, stroke: red\n linkStyle 2 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 3 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 4 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 5 stroke-width: 4px, stroke: red\n linkStyle 8 stroke-width: 4px, stroke: red\n linkStyle 9 stroke-width: 4px, stroke: red\n linkStyle 10 stroke-width: 4px, stroke: red\n linkStyle 11 stroke-width: 4px, stroke: red\n linkStyle 12 stroke-width: 4px, stroke: red\n linkStyle 13 stroke-width: 4px, stroke: red\n linkStyle 14 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 15 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 16 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 17 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n linkStyle 18 stroke-width: 4px, stroke: yellow, stroke-dasharray: 5\n```\n\n\u003c/td\u003e\n\u003ctd width=\"480px\"\u003e\n\n```mermaid\nflowchart TB\n style HOSTING_PROVIDER fill:#4d683b,color:#fff\n style DDNS_PROVIDER fill:#69587b,color:#fff\n style INTERNET_SERVICE_PROVIDER fill:#205566,color:#fff\n style INTERNET_SERVICE_PROVIDER2 fill:#205566,color:#fff\n style SINGLE_BOARD_COMPUTER fill:#665151,color:#fff\n style CONTAINER_ENGINE fill:#664343,color:#fff\n style TRAEFIK_CONTAINER fill:#663535,color:#fff\n style MYAPP_CONTAINER fill:#663535,color:#fff\n style TRAEFIK_ROUTER fill:#806030,color:#fff\n style TRAEFIK_MIDDLEWARE fill:#806030,color:#fff\n style DNS_RESOLVER fill:#805060,color:#fff\n DOMAIN(example.com)\n SUBDOMAIN_MYAPP(myapp.example.com)\n DDNS(myddns.ddns.net)\n ROUTER_PUBLIC_IP[public IP]\n ROUTER_PORT80{{80/tcp}}\n ROUTER_PORT443{{443/tcp}}\n ROUTER2_DNS[DNS]\n DOCKER_TRAEFIK_PORT443{{433/tcp}}\n DOCKER_TRAEFIK_PORT80{{80/tcp}}\n DOCKER_MYAPP_PORT{{port/tcp}}\n TRAEFIK_ROUTER_MYAPP(myapp.example.com)\n TRAEFIK_MIDDLEWARE_REDIRECT(HTTPS redirect)\n ROOT_DNS_SERVERS[Root DNS servers]\n CLOUDFLARE(Cloudflare, etc.)\n\n subgraph HOSTING_PROVIDER[DOMAIN NAME REGISTRAR]\n DOMAIN\n SUBDOMAIN_MYAPP\n end\n\n subgraph DDNS_PROVIDER[DYNAMIC DNS PROVIDER]\n DDNS\n end\n\n subgraph INTERNET_SERVICE_PROVIDER[ISP ROUTER]\n ROUTER_PUBLIC_IP\n ROUTER_PORT80\n ROUTER_PORT443\n end\n\n subgraph INTERNET_SERVICE_PROVIDER2[CLIENT ISP ROUTER]\n ROUTER2_DNS\n end\n\n subgraph DNS_RESOLVER[DNS RESOLVER]\n CLOUDFLARE\n end\n\n subgraph SINGLE_BOARD_COMPUTER[BANANA PI M5]\n subgraph CONTAINER_ENGINE[DOCKER]\n subgraph MYAPP_CONTAINER[MYAPP CONTAINER]\n DOCKER_MYAPP_PORT\n end\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyann39%2Fself-hosted","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fyann39%2Fself-hosted","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fyann39%2Fself-hosted/lists"}