{"id":13588098,"url":"https://github.com/dniel/blogr-pve","last_synced_at":"2025-04-08T02:35:08.596Z","repository":{"id":104173502,"uuid":"59731716","full_name":"dniel/blogr-pve","owner":"dniel","description":"Puppet provisioning of HA failover/cluster environment implemented in Proxmox Virtual Environment and Linux boxes.","archived":true,"fork":false,"pushed_at":"2019-11-21T13:29:57.000Z","size":104120,"stargazers_count":28,"open_issues_count":8,"forks_count":8,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-08-02T16:09:01.203Z","etag":null,"topics":["centralized-logging","centralized-monitoring","consul","docker","grafana","high-availability","lxc","microservices-architecture","prometheus","proxmox","puppet","service-discovery","traefik","virtualization"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dniel.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2016-05-26T08:03:32.000Z","updated_at":"2023-08-03T21:26:49.000Z","dependencies_parsed_at":"2023-07-04T16:16:31.174Z","dependency_job_id":null,"html_url":"https://github.com/dniel/blogr-pve","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dniel%2Fblogr-pve","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dniel%2Fblogr-pve/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dniel%2Fblogr-pve/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dniel%2Fblogr-pve/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dniel","download_url":"https://codeload.github.com/dniel/blogr-pve/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223300734,"owners_count":17122679,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["centralized-logging","centralized-monitoring","consul","docker","grafana","high-availability","lxc","microservices-architecture","prometheus","proxmox","puppet","service-discovery","traefik","virtualization"],"created_at":"2024-08-01T15:06:30.561Z","updated_at":"2024-11-06T07:30:58.011Z","avatar_url":"https://github.com/dniel.png","language":"Shell","funding_links":[],"categories":["Shell","prometheus"],"sub_categories":[],"readme":"# Proxmox Virtual Environment\r\n#### Puppet scripts for my home lab high-availability(HA) development environment.\r\nMy home lab environment for software development and infrastructure\r\nthat has a production like configuration that I use to learn and\r\npractice on DevOps and software development. The setup is similar\r\nto and handle some of the complexity that a IT-department would\r\nneed to meet the demands of a production, test and dev environment.\r\n\r\nIt configures a pretty standard high-availability environment with a\r\npair of reverse proxies that load balance in front of four backend\r\nservers that has been divided in two clusters and using a primary\r\nand hot-standby for stored state.\r\n\r\nThe network has been segmented into subnetworks for dmz and a internal\r\ncorporate network + more to emulate a production like environment.\r\n\r\n#### Features\r\n* Clustered proxmox virtual environment on three machines.\r\n* Hardware failover with virtualized linux containers deployed on the proxmox cluster.\r\n* Segmented network, divided into subnets and vlans for security\r\n* Load balanced frontend servers.\r\n* Load balanced node backend server.\r\n* A sample web application.\r\n* Immutable lightweight lxc containers with one dedicated task each. (debian8)\r\n* Service discovery with consul, health check of nodes and services with consul\r\n* Provisioning with puppet scripts for configuration and containers\r\n* Centralized logging from all nodes to ELK stack\r\n* Notifications of events to mattermost\r\n* Continuous deployment, a build pipeline with Jenkins2\r\n\r\n## Hardware\r\nThree retired work laptops, Lenovo W520 and W530 with 16GB ram\r\n and old school spinning hard drives connected with a Cisco\r\n switch with VLAN support, [Cisco SG 200-08](https://www.amazon.co.uk/Cisco-SG200-08-8-port-Gigabit-SLM2008T-EU/dp/B004UOT4BI/ref=sr_1_2?ie=UTF8\u0026qid=1473971098\u0026sr=8-2\u0026keywords=SG+200-08), for directing network\r\n traffic between them.\r\n\r\n![alt tag](doc/machines.jpg)\r\n\r\n## Proxmox cluster on three nodes.\r\nA screenshot from the Proxmox Web Admin. The three machines has\r\nbeen added to the same cluster and can be managed through the\r\nsame admin console.\r\n\r\n![alt tag](doc/pve.jpg)\r\n\r\n## Virtual machines overview with networks\r\nThis is how is looks like if modelled with Archimate and Archi.\r\nI have modelled just the virtual machines, the networks and firewalls\r\nthat connect them.\r\n\r\n![alt tag](doc/overview.png)\r\n\r\n## The sample application that is deployed\r\nIs a very simple React application [Blogr](https://github.com/dniel/blogr-workshop)\r\nthat uses a Node Express backend and a PostgreSQL database for\r\npersistence. Right now its deployed using the same puppet script\r\nthat install the configuration and system software but this\r\neventually be handled by Jenkins Pipeline.\r\n\r\n## Configuration\r\n#### [Network configuration](doc/setup_network.md) with [Pfsense](doc/setup_pfsense.md)\r\n#### Container virtualization with [Linux LXC](doc/setup_lxc.md) and [Proxmox](doc/setup_proxmox.md) \r\n#### [Continuous Delivery with Jenkins Pipeline](doc/setup_jenkins.md)\r\n#### [Service Discovery with Consul](doc/setup_consul.md)\r\n#### Application orchestration with Rancher, TODO\r\n#### Centralized logging with ELK, TODO\r\n#### Centralized monitoring with Prometheus and Grafana, TODO\r\n#### Provisioning with Puppet, TODO\r\n\r\n## Limitations\r\n* Tested with Proxmox Virtual Environment 4.2-2/725d76f0.\r\n* Virtual containers in Proxmox LXC installed with Debian 8.\r\n* pfSense for firewall, gateway, routing, virtual ips of WAN, DMZ and LAN.\r\n* A hardware network switch to route the vlans between different hardware nodes.\r\n* Will not configure proxmox or pfsense, both must be manually installed and configured.\r\n\r\n### Future development\r\n* SNMP monitoring of all hosts and services, Grafana.\r\n* Central user store, LDAP? \r\n* Automatic backup of whole cluster.\r\n* Store secrets in HashiCorp Vault\r\n* Reimplement - Firewall with failover. (removed, will be re-added when other stuff has been finished)\r\n* Reimplement - Database with failover. (removed, will be re-added when other stuff has been finished)\r\n* Create base default image for LXC containers containing bootstrap code for faster creation of new containers.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdniel%2Fblogr-pve","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdniel%2Fblogr-pve","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdniel%2Fblogr-pve/lists"}