{"id":15489746,"url":"https://github.com/theandrew168/devops","last_synced_at":"2025-04-22T18:28:51.923Z","repository":{"id":47768812,"uuid":"291771952","full_name":"theandrew168/devops","owner":"theandrew168","description":"Automation for personal projects and websites","archived":false,"fork":false,"pushed_at":"2025-04-03T21:41:06.000Z","size":525,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-19T13:09:43.031Z","etag":null,"topics":["ansible","devops","digitalocean","infrastructure","terraform"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/theandrew168.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-08-31T16:48:34.000Z","updated_at":"2025-04-03T21:41:10.000Z","dependencies_parsed_at":"2024-01-28T05:27:01.511Z","dependency_job_id":"2c90b486-0e8d-46ca-b609-4daebe6c4a6b","html_url":"https://github.com/theandrew168/devops","commit_stats":{"total_commits":359,"total_committers":6,"mean_commits":"59.833333333333336","dds":0.2924791086350975,"last_synced_commit":"624f4d2ec6dd75cf677967aa1642870d211e4db2"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fdevops","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fdevops/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fdevops/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/theandrew168%2Fdevops/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/theandrew168","download_url":"https://codeload.github.com/theandrew168/devops/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250296705,"owners_count":21407073,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","devops","digitalocean","infrastructure","terraform"],"created_at":"2024-10-02T07:07:34.238Z","updated_at":"2025-04-22T18:28:51.901Z","avatar_url":"https://github.com/theandrew168.png","language":"HCL","readme":"# devops\n\nThis repo contains the automation I use for my personal projects and websites.\nTwo open source tools carry most of the weight: [Ansible](https://www.ansible.com/) and [Terraform](https://www.terraform.io/).\nI currently use [DigitalOcean](https://www.digitalocean.com/) to host all of my infrastructure but this setup can be easily adapted to other hosting providers.\n\n## Scheduling\n\nIn general, the infra managed by this repo operates on the following schedule.\nNote that not all of these events occur each day.\nSystem restarts, for example, only occur as needed (when a package or kernel update requires it).\n\n| What             | When                      |\n| ---------------- | ------------------------- |\n| System restarts  | 8:00 AM UTC (2:00 AM CST) |\n| Database backups | 9:00 AM UTC (3:00 AM CST) |\n\n## Install\n\nFirst, follow the corresponding guides for [installing Ansible](https://docs.ansible.com/ansible/latest/installation_guide/intro_installation.html) and [installing Terraform](https://www.terraform.io/downloads.html).\nYou can verify that each tool is installed correctly by checking their versions:\n\n```\nansible --version\nterraform version\n```\n\n## Setup\n\nIf you are reading this then we must be collaborating on something.\nJust reach out to me and I'll get you the actual credentials.\nI like to export these environment variables in `$HOME/.profile` but you can put them wherever you prefer.\nJust remember to either source the file after adding the vars or log out and back in again.\n\n### Terraform Backend\n\nSince Terraform's `.tfstate` files may contain sensitive information, a [DigitalOcean Space](https://www.digitalocean.com/products/spaces/) is used to keep them private.\nTerraform views DigitalOcean Spaces as an S3 backend so the following environment variables (also obtained from the web console) are necessary to hook everything up:\n\n```\nexport AWS_ACCESS_KEY_ID=\"spaces_key_from_web_console\"\nexport AWS_SECRET_ACCESS_KEY=\"spaces_secret_from_web_console\"\n```\n\n### Terraform DigitalOcean Access\n\nTerraform needs an access token in order to talk to DigitalOcean.\nThis token can be obtained from DigitalOcean's web console:\n\n```\nexport DIGITALOCEAN_TOKEN=\"access_token_from_web_console\"\n```\n\nAdditionally, a few more vars are needed in order to interact with DigitalOcean Spaces (S3-compatible object storage):\n\n```\nexport SPACES_ACCESS_KEY_ID=\"spaces_key_from_web_console\"\nexport SPACES_SECRET_ACCESS_KEY=\"spaces_secret_from_web_console\"\n```\n\n### Terraform Init\n\nThe only thing left to do now is initialize Terraform for each project:\n\n```\ncd infra/\u003cproject\u003e/\nterraform init\n```\n\n### Ansible Vault Password\n\nSince Ansible is occasionally used to deploy encrypted values, a password is needed in order to decrypt them.\nThrow the correct password (ask an existing team member) into `$HOME/.vault_pass.txt` and export this environment variable which points to it's location:\n\n```\nexport ANSIBLE_VAULT_PASSWORD_FILE=\"$HOME/.vault_pass.txt\"\n```\n\n## Usage\n\nIn general, setting up a server for app hosting involves two steps:\n\n1. use Terraform to create the droplet and associated hostname\n2. use Ansible to configure the server\n\nThe Ansible playbook only needs to be executed once (when the droplet is initially created) to setup the SSH server config, firewall rules, unattended ugprades, etc.\n\n### Terraform\n\nThe Terraform workflow is quite simple: use the [plan command](https://www.terraform.io/docs/cli/commands/plan.html) to see what changes are pending and then the [apply command](https://www.terraform.io/docs/cli/commands/apply.html) to apply them.\nThat last `refresh-only` command picks up default tags and other resource nuances added automatically after initial creation.\n\n```\ncd infra/\u003cproject\u003e/\nterraform plan\nterraform apply\nterraform apply -refresh-only\n```\n\n### Ansible\n\nNext, we need to run Ansible to get the server ready for hosting an application.\nNote that this only has to happen once when the droplet is first created.\n\nThe first time Ansible is ran against a new server, it must be done as `root`.\nHowever, the first tasks executed create at least one admin user that must be used for all future invocations.\nThe baseline [server role](https://github.com/theandrew168/devops/tree/master/roles/server) will abort after admin creation if the Ansible user is still `root`.\n\nFirst run against a new server:\n\n```\nansible-playbook -i hosts -u root foobar.yml\n```\n\nAfter manually logging into a server and setting your password:\n\n```\nansible-playbook -i hosts -u \u003cuser\u003e --ask-become-pass foobar.yml\n```\n\n## Upgrades\n\n### Ubuntu Server\n\nThe process of performing a major Ubuntu Server upgrade in-place (22.04 to 24.04, for example) is fairly straightfoward.\nIf the server uses a PostgreSQL database and [pg2s3](https://github.com/theandrew168/pg2s3) for backups, take a backup before upgrading:\n\n```\npg2s3 -conf /etc/pg2s3.conf backup\n```\n\nThen run the following commands:\n\n```\napt update\napt upgrade\napt dist-upgrade\ndo-release-upgrade\n```\n\nAfterward, I like to re-run the Ansible playbook for the server to ensure everything is configured correctly.\n\n### PostgreSQL\n\nOften, upgrading Ubuntu Server will include a newer version of PostgreSQL.\nUsing this newer database version requires a few extra steps:\n\n```\n# stop pg2s3 while performing the upgrade\nsystemctl stop pg2s3\n# stop the server's main app service (whichever one uses the DB)\nsystemctl stop \u003cservice\u003e\n# list available PG clusters (should be one old and one new)\npg_lsclusters\n# stop ALL clusters\npg_ctlcluster \u003cverison\u003e \u003cname\u003e stop\n# delete the existing database data\nrm -fr /mnt/\u003cdatabase_data_dir\u003e/data\n# re-run the ansible automation to configure the latest version's cluster\n# if you want, you can comment out all but the \"postgresql\" role here\nansible-playbook -i hosts -u \u003cuser\u003e --ask-become-pass \u003cservice\u003e.yml\n# restore the latest backup via pg2s3\npg2s3 -conf /etc/pg2s3.conf restore\n# start the service\nsystemctl start \u003cservice\u003e\n# start pg2s3\nsystemctl start pg2s3\n# drop the old database cluster\npg_dropcluster \u003cversion\u003e \u003cname\u003e\n# restart the server to ensure everything starts up correctly\nreboot\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheandrew168%2Fdevops","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftheandrew168%2Fdevops","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftheandrew168%2Fdevops/lists"}