{"id":19565531,"url":"https://github.com/swaldman/mchange-sysadmin-scripts","last_synced_at":"2025-02-26T09:24:28.031Z","repository":{"id":191555726,"uuid":"684875349","full_name":"swaldman/mchange-sysadmin-scripts","owner":"swaldman","description":"systemd-managed scala scripts for sysadmin tasks","archived":false,"fork":false,"pushed_at":"2024-03-27T03:41:03.000Z","size":3204,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-09T00:07:41.061Z","etag":null,"topics":["scala","script","sysadmin","sysadmin-scripts","sysadmin-tasks","systemd"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/swaldman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-30T03:08:41.000Z","updated_at":"2023-09-09T02:04:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"922454b2-f763-4eb0-8601-afa3177b27e6","html_url":"https://github.com/swaldman/mchange-sysadmin-scripts","commit_stats":null,"previous_names":["swaldman/mchange-sysadmin-scripts"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swaldman%2Fmchange-sysadmin-scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swaldman%2Fmchange-sysadmin-scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swaldman%2Fmchange-sysadmin-scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/swaldman%2Fmchange-sysadmin-scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/swaldman","download_url":"https://codeload.github.com/swaldman/mchange-sysadmin-scripts/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":240824190,"owners_count":19863615,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["scala","script","sysadmin","sysadmin-scripts","sysadmin-tasks","systemd"],"created_at":"2024-11-11T05:27:24.412Z","updated_at":"2025-02-26T09:24:27.964Z","avatar_url":"https://github.com/swaldman.png","language":"Scala","readme":"# mchange-sysadmin-scripts\n\nThis is an experiment in building sysadmin scripts in `Scala` using [`scala-cli`](https://scala-cli.virtuslab.org/) and `systemd`.\nIt makes use of the library [`mchange-sysadmin-scala`](https://github.com/swaldman/mchange-sysadmin-scala) for a variety of utilities,\nin particular for its [`TaskRunner`](https://github.com/swaldman/mchange-sysadmin-scala/blob/main/sysadmin/src/com/mchange/sysadmin/TaskRunner.scala)\nwhich models a sysadmin task as a pipeline of steps.\n\nThe main reason to do this is that I can write cleaner, surer code in Scala than I am capable of beyond very simple shell scripts.\n\nIt's also easy to add features like detailed reporting in nicely-formatted, color-coded HTML mail:\n\n![Success and failure e-mails, side-by-side](doc/media/backup-postgres-side-by-side-small.png)\n\n([success.pdf](doc/media/backup-postgres-succeeded.pdf), [failure.pdf](doc/media/backup-postgres-failed.pdf))\n\nOf course, plaintext reports are also generated for regular logs (and as a `text/plain` alternate in the e-mails).\n\nWhile you may find some of these scripts usable as-is, once you understand the set-up, it is easy to\ncustomize and to write your own scala scripts that `systemd` will trigger and that will e-mail nice reports.\n\n## Prerequisites\n\n[`scala-cli`](https://scala-cli.virtuslab.org/) must be installed and available on `root`'s path.\n\nThis application is intended for linux machines running [`systemd`](https://systemd.io/).\n\n## Installation\n\n### I. Create a user named `mchange-sysadmin`\n\nWhen possible, admin tasks run as that user rather than as `root`.\nHowever, some tasks cannot run as `mchange-sysadmin`, must run as `root`, or some specific user like `nginx`.\n\n\n### II. Clone this repository as `mchange-sysadmin`\n\nIt doesn't matter where so much. But this will be where your scripts and systemd units live their best, permanent\nlives, so give it a few minutes of thought.\n\n(If you want scripts as a particular version or tag, just use `git checkout \u003ctag-or-commit\u003e` to get the version you want.)\n\n### III. Configure the environment\n\n1. Make the config directory. This directory will contain e-mail and perhaps database credentials, so set restrictive permissions.\n   This directory, and its environment file,  should be accessible only to `mchange-sysadmin`. \n   ```plaintext\n   # mkdir /etc/mchange-sysadmin/\n   # chmod go-rwx /etc/mchange-sysadmin/\n   # chown mchange-sysadmin:mchange-sysadmin /etc/mchange-sysadmin\n   ```\n\n2. Set up the file `/etc/mchange-sysadmin/mchange-sysadmin.env`:\n\n   Note that the backup-database scripts interpret destinations containing a `:` as [rclone](https://rclone.org/) destinations.\n   \n   ```plaintext\n   MCHANGE_SYSADMIN_SCRIPTS_HOME= # The directory into which you cloned this distribution\n   SYSADMIN_MAIL_FROM=            # The e-mail address sysadmin mail should be sent from\n   SYSADMIN_MAIL_TO=              # The e-mail address sysadmin mail should be sent to\n\n   # Optional\n   PG_BACKUPS_DEST=               # If you'll use the backup-postgres script, an rclone destination to which to send backups\n   MYSQL_BACKUPS_DEST=            # If you'll use the backup-mysql script, an rclone destination to which to send backups\n\n   # Authentication resources -- probably use these as is!\n   SMTP_PROPERTIES=/etc/mchange-sysadmin/smtp.properties\n   RCLONE_CONFIG=/etc/mchange-sysadmin/rclone.conf\n   PGPASSFILE=/etc/mchange-sysadmin/pgpass\n   MYSQL_DEFAULTS_EXTRA=/etc/mchange-sysadmin/mysql-root@localhost.cnf\n   ```\n\n   This file should also be owned by, and read-only by, `mchange-sysadmin:mchange-sysadmin`.\n\n### IV. Provide authentication resources\n\nYou'll need to provide SMTP connection and authentication information. While it is possible to\ndo that via [environment variables](https://github.com/swaldman/mchange-sysadmin-scala/blob/main/src/com/mchange/sysadmin/Smtp.scala) as above,\nprocess environment can leak and is insecure. So it's best to\nonly specify the location of an `smtp.properties` file.\n\n   ```plaintext\n   mail.smtp.user=                     # Your e-mail provider username\n   mail.smtp.password=                 # Your e-mail provider password\n   mail.smtp.host=                     # Your e-mail provider's SMTP host\n   mail.smtp.port=465                  # Your e-mail provider's SMTP port (if 587, probably set mail.smtp.starttls.enable to true)\n\n   ## less commonly\n   #mail.smtp.port=587\n   #mail.smtp.starttls.enable=false\n   #mail.smtp.debug=false\n   ```\n\nDepending on which scripts you run, and whether you use `rclone` backup destinations, you may need to set up\nthe following files:\n\n   * `/etc/mchange-sysadmin/rclone.conf` \u0026mdash; this file defines `rclone` destinations\n     and authentication thereto, if you use `rclone` destinations in backup scripts.\n     One way to get it is just to use `rclone config` and let that generate the config file\n     as `~/.config/rclone/rclone.conf`, then copy that to `/etc/mchange-sysadmin/rclone.conf`.\n\n     It may work to `export RCLONE_CONFIG=/etc/mchange-sysadmin/rclone.conf`, then run `rclone config`, but I haven't\n     tried it yet.\n     \n   * `/etc/mchange-sysadmin/rclone.conf/pgpass` \u0026mdash; In order for user `mchange-sysadmin` to authenticate as super-user `postgres`,\n     you will want to\n        1. [Set a password](https://chartio.com/resources/tutorials/how-to-set-the-default-user-password-in-postgresql/) for superuser `postgres`\n        2. Verify that your `pg_hba.conf` allows access from localhost with password authentication\n           (e.g. `scram-sha-256`)\n        3. Create a `/etc/mchange-sysadmin/rclone.conf/pgpass` file of the form\n           ```\n           127.0.0.1:5432:*:postgres:\u003csuperuser-postgres-password\u003e\n           ```\n           You need access to ALL databases. See the [postgres docs](https://www.postgresql.org/docs/current/libpq-pgpass.html).\n\t   \n   * `/etc/mchange-sysadmin/mysql-root@localhost.cnf`, which should [look like](https://stackoverflow.com/questions/34916074/how-to-pass-password-from-file-to-mysql-command)\n      ```\n      [client]\n      password=\"\u003cyour-mysql-root-password\u003e\"\n      ```\n\nAll these files should be owned by `mchange-sysadmin`, and `chmod 600`.\n\t   \n\n### V. Link service and timer unit files where `systemd` will find them\n\nFor example, if you want to use the `backup-postgres` script and you've cloned this distribution into `/usr/local`, then...\n\n```plaintext\n# cd /etc/systemd/system\n# ln -s /usr/local/mchange-sysadmin-scripts/systemd/backup-postgres.service\n# ln -s /usr/local/mchange-sysadmin-scripts/systemd/backup-postgres.timer\n```\n\nOf course, review the unit files, and edit them to suit. Perhaps you want postgres backed up more frequently, or less.\n\n### VI. Test your service\n\nIt's just...\n\n```plaintext\n# systemctl start backup-postgres\n```\n\nTo follow what's happening, and debug any problems:\n\n```plaintext\n# journalctl -u backup-postgres --follow\n```\n\nOnce the script runs cleanly, you should see a report in the log, and receive a prettier one by e-mail\nat the `SYSADMIN_MAIL_TO` address you've configured.\n\n### VII. Install and start your timer\n\nFor every script you want to be triggered automatically, you'll need to install and start a timer:\n\n```plaintext\n# systemctl enable backup-postgres.timer\n# systemctl start backup-postgres.timer\n```\n\nOnce that's done, verify the timer is set:\n\n```plaintext\n# systemctl status backup-postgres.timer\n● backup-postgres.timer - schedule a weekly postgres backup\n     Loaded: loaded (/etc/systemd/system/backup-postgres.timer; enabled; vendor preset: enabled)\n     Active: active (waiting) since Tue 2023-09-05 18:24:40 UTC; 2 days ago\n    Trigger: Mon 2023-09-11 06:10:39 UTC; 2 days left\n   Triggers: ● backup-postgres.service\n```\n\nIf there is a `Trigger` time, the timer is running.\n\nThat's it!\n\nLink/test/install whichever scripts you want.\n\n## Write your own scripts\n\nScala scripts that organize administrative tasks intio pipelines live in the [taskbin](taskbin) directory.\nCheck [`snapshot`](taskbin/snapshot) for a very simple example.\nSee the documentation for [mchange-sysadmin-scala](https://github.com/swaldman/mchange-sysadmin-scala) for details on\nhow to define a `TaskRunner` and its tasks.\n\nJust write a script \u0026mdash; which can and often does require command line arguments and/or environment vars \u0026mdash;\nto execute your task, using the default reporters.\n\nOnce you have defined your task, check out the `.service` and `.timer` files in [`systemd`](systemd).\nIt's very easy to follow the pattern and make new ones for your new task. \n\nYou can define traditional shell scripts as helpers, and place them in the [`bin`](bin) directory.\nVery trivial shell scripts can dramatically simplify the Scala you might othewise need to write.\n\nYour systemd services should run\nas user `mchange-sysadmin` if privileged access is not required, or as `root` if it is,\nbecause only those two users have access to configuration and authentication information.\n\n(If you want, you can play around with defining a shared group for other unprivileged accounts,\nand making config and auth files group readable. Note, though, that all users that run\nmchange-sysadmin scripts will download their own copies of its scala dependencies, costing about\n500 MB per user as of this writing.)\n\n## Known shortcomings\n\nFor security purposes, scripts run as the least-privilged user they can\nrun under, not always as `root`. However, `scala-cli` will download dependencies\n(including JVMs!) and generate compilation artifacts on a per-user basis.\nEvery user that runs a script ends up with a variety of duplicated infrastructure\nburied in its home directory.\n\nThe storage overhead is **substantial**, 500M to 700M per user. Perhaps it would\nbe best to restrict to users `mchange-sysadmin` and `root`, so we don't bear\nthis cost too many times?\n\nIt's a tradeoff of space vs a small amount of additional security.\n\n## Contributing\n\nConsider forking and maintaining a branch for your own additions and customizations.\n\nPull requests that include additions that might be broadly useful \u0026mdash; or enhancements of or fixes to what is already here \u0026mdash;\nwould be greatly appreciated.\n\n## Miscellaneous notes\n\nFor a list of supported timezones, see `timedatectl list-timezones`.\n\n---\n\n_Note: This project is distinct from the venerable [mchange-admin](https://github.com/swaldman/mchange-admin) project, which is mostly\nabout ad-hoc package management._\n\n\n\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswaldman%2Fmchange-sysadmin-scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fswaldman%2Fmchange-sysadmin-scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fswaldman%2Fmchange-sysadmin-scripts/lists"}