{"id":27258267,"url":"https://github.com/buanzo/hume","last_synced_at":"2025-09-11T12:34:56.320Z","repository":{"id":74580359,"uuid":"254244164","full_name":"buanzo/hume","owner":"buanzo","description":"Hume is a tool to aid sysadmins in their quest to write better system administration scripts by enhancing instrumentation of scripts (TL;DR it lets you query the status/info of a certain sysadmin task)","archived":false,"fork":false,"pushed_at":"2020-08-09T12:33:48.000Z","size":182,"stargazers_count":16,"open_issues_count":0,"forks_count":4,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-10T01:55:09.099Z","etag":null,"topics":["agnostic","bash","bash-scripting","cron","devops-teams","devops-tools","instrumentation","python-3","sysadmin","sysadmin-tasks","sysadmin-tool","system-administration"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/buanzo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-04-09T01:46:59.000Z","updated_at":"2022-09-03T20:39:39.000Z","dependencies_parsed_at":"2023-10-05T05:38:08.397Z","dependency_job_id":null,"html_url":"https://github.com/buanzo/hume","commit_stats":{"total_commits":93,"total_committers":3,"mean_commits":31.0,"dds":"0.021505376344086002","last_synced_commit":"7747e5112ec3bbc97397c73e4ef2e2f0de4d6140"},"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buanzo%2Fhume","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buanzo%2Fhume/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buanzo%2Fhume/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buanzo%2Fhume/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/buanzo","download_url":"https://codeload.github.com/buanzo/hume/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248333907,"owners_count":21086277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agnostic","bash","bash-scripting","cron","devops-teams","devops-tools","instrumentation","python-3","sysadmin","sysadmin-tasks","sysadmin-tool","system-administration"],"created_at":"2025-04-11T03:20:10.701Z","updated_at":"2025-09-11T12:34:56.309Z","avatar_url":"https://github.com/buanzo.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# hume\nImagine this: You have some cron scripts. They need to run correctly, and you need to know\nif anything happens. You usually check logs via email or some cool\ndashboard. Yeah. But it gets boring, repetitive and you end up not checking\nthe logs. It is a well known sysadmin syndrome.\n\nHume client works and can be used in bash scripts, or during a screen\nsession, to provide real time updates to your managers, customers, etc.\n\nSo, for instance write those scripts adding hume commands in them:\n\n```\n#!/bin/bash\n# This script updates wordpress core, plugins and themes\n# on a number of different directories.\nwpcli='/usr/local/bin/wp'\n\n# Maybe hume could read this variables\nHTASK=\"WORDPRESS_UPDATE\"\nHTAGS=\"wordpress,cron,updates\"\n\nhume -c counter-start -t \"$HTASK\" -T \"$HTAGS\" -m \"Starting wordpress update cron process\"\nfor dir in \"/var/www/site1/htdocs\" \"/var/www/site2/htdocs\" \"/var/www/site3/htdocs\"\ndo\n\thume -t \"$HTASK\" -T \"$HTAGS\" -L info -m \"Updating in $dir\"\n        cd $dir || hume -t \"$HTASK\" -T \"$HTAGS\" -L error -m \"$dir does not exist\"\n        $wpcli core update || hume -t \"$HTASK\" -T \"$HTAGS\" -L error -m \"Core update error in $dir\"\n        $wpcli plugin update --all || hume -t \"$HTASK\" -T \"$HTAGS\" -L error -m \"Plugins update error in $dir\"\n        $wpcli theme update --all || hume -t \"$HTASK\" -T \"$HTAGS\" -L error -m \"Themes update error in $dir\"\n\thume -t \"$HTASK\" -T \"$HTAGS\" -L info -m \"Update process for $dir finished\"\ndone\nhume -c counter-stop -t \"$HTASK\" -T \"$HTAGS\" -m \"Finished wordpress update cron task\"\n```\n\nThe above usage example gives you a general idea that Hume is an agnostic system administration tool/framework.\n\n# Why is Hume agnostic?\n\nEvery sysadmin, believe it or not, is a human being.\n\nThe same reason there are awesome tools, for example Faraday (used by\npenetration testers and other members of the information security community)\nallows each individual pentester to do their work using the\ntools they like, and still be able to structurally share and enhance the data\nthat comes out of them in a centralized knowledge database.\n\nHume is the system administration equivalent of Faraday. And I think it goes\nhand in hand with devops, devsecops, etc.\n\nAnd both are agnostic in relation to how each person does their job: you do\nNOT need to use bash even. If you know enough Python, you can import hume\ninto your script and report directly from your code. TODO: add an example.\n\n# Current Status / Needs\n\n* Query database and query tools (Codename KANT).\n\nHumed works in localhost mode only, but primary/secondary ideas have been\nconsidered (see below) - There is NO QUERY TOOL, but you can get data off\nthe humed.sqlite3 database\n\nThen you could check the status of the latest run of the task:\n\n```\n#!/bin/bash\n# echo check status of wordpress_update task for webserver host\nhumequery -t WORDPRESS_UPDATE --latest --hostname=\"webserver\"\n```\n\nAnd you would get the list of every hume event, plus a summary, including \n\n* We need a UI/UX designer/developer, maybe a REST API for querytools.\n\n\n# Slack support\n\nHume can send messages to Slack using the Incoming Webhooks method. Create\na channel for hume messages, then a Slack App, finally create an incoming\nwebhook for that channel and paste the webhook url into the humed\nconfiguration file.\n\nMessages may be routed to different Slack channels by error level. The\n`webhook_default` setting is used as fallback.  You can also map specific\ntasks to dedicated channels using the `task_channels` dictionary in the\n`slack` section.  Unmatched tasks fall back to the level-based routing or\n`webhook_default`.\n\nExample:\n\n```yaml\nslack:\n    webhook_default: https://hooks.slack.com/services/XXX/YYY/ZZZ\n    task_channels:\n        backup: https://hooks.slack.com/services/AAA/BBB/CCC\n        deploy: https://hooks.slack.com/services/DDD/EEE/FFF\n```\n\n# Message format and validation\n\nEach hume message includes a `version` field that indicates the payload\nformat. The daemon validates every incoming packet, checking the version\nand required fields before accepting it. Messages that fail validation are\nignored to prevent malformed data from being stored.\n\n# Implementation, concepts, ideas\n\n## Ideas for implementation\n\nhume uses zeromq over loopback to connecto to a hume daemon on the same\nserver. It must reduce its footprint as much as possible, we dont want hume\nto slow down other processes. I am testing zmq-ipc and zmq-tcp, as well as\nPUSH/PULL and REQ/REP scenarios. A systemd unit file is included for the hume\ndaemon (humed). A pid file will be read and checked by the hume client to\nknow beforehand if the hume daemon is working. A watchdog script that\ntriggers a special hume mode should exist to report working status. A lack\nof watchdog-pinging will help the devops team detect the situation quickly.\n\nThe hume daemon will filter/store the hume packets, and send them to the\nmain repository which will be a relational database coupled with redis for\ncurrent-data storage and querying.\n\n## Weird future concepts\n\n* Signed script verification (imagine, the script is created, signed, hume\nchecks the signature and executes/notifies if signature does not verify)\n\n* Of course, authentication and authorization will need to be implemented.\n\n* task templates: imagine this: 'hume template get wordpress_backup': You\nget a wordpress_backup bash script that's only the required hume tasknames,\nrequired informational stages.  All comments.  You just write the script to\nfollow the devops pattern.\n\n* watchdog command: 'This task needs to run daily' -\u003e cron failed? hume\ncommand that lets the watchdog know it is effectively running daily.\n\n* A publisher/subscriber subsystem?\n\nMaybe one wants to propagate an event.\n\nMaybe we want to set some value for others to consume (this could be related to Prometheus idea below)\n\n* Prometheus compatibility\n\nHumed should maintain the last status regarding a particular host:task\n(humed might end up receiving information from more than one host, specially\nin containerized environments?).\n\nThat data is valid for a time series based scraper such as Prometheus.\n\nAn in-memory db should work in this scenario, with possibly a persistent option.\n\nKEY = host:task?\nVALUES = All of HumePkt? timestamp included.\n\nThe MSG might not end up being graphed... :P but Grafana for instance supports\nText panels. It might be nice to be able to design a Hume Tasks dashboard.\n\n`humed` exposes a simple HTTP endpoint at `/metrics` which provides\nPrometheus-formatted statistics.  When the optional `token` setting is\ndefined under the `metrics` section of the configuration, requests must\ninclude an `Authorization` header with the value `Bearer \u003ctoken\u003e` in order\nto retrieve the metrics.\n\nThe daemon can also require an authentication token from `hume` clients. Set\n`auth_token` at the top level of `humed`'s configuration and pass the same value\nusing the `--auth-token` option (or `HUME_TOKEN` environment variable) when\ninvoking `hume`.\n\n* mailx drop-in replacement\n\nunattended-upgrades and other packages send mail notifications using mailx. Write\na hume-mailx that can be used as alternative.\n\n# Components\n\n## hume\n\ncalled from within scripts.  writes to a local publish queue.  maybe a\nfallback to sqlite that humed can read on startup?\n\n## humed\n\nits the local publish queue.  consumes messages, applies\nfiltering/preprocessing, security, etc.  Can be set to primary or secondary mode. \nThis way the hume CLI tool will just push gather some info, then push zmq\nmessages to localhost humed.  if localhost humed is a secondary, it will have a\nthread to send pending hume messages to the primary.  query tool should work\nagainst secondary or primary.\n\n## Transfer Methods\n\nHumed currently supports slack, logstash, syslog and remote syslog.\n\nNext feature is multiple transfers, and this will require a bit of\nrefactoring, plus a threaded model.\n\nIn the future it will probably support fluentd and kant (our own query\ndatabase / tool / dashboard system).\n\nDevelopment will continue with logstash, then kant and finally fluentd,\nbecause I am not finding good fluent client/agent implementations in\npython3.\n\nThe logstash support uses python-logstash-async.\n\n\n# DEVELOPMENT NOTES\n\n## Define Basic CLI usage\n* register and start process execution\n* add events to process --warn, --info, --error, --debug\n* event flagging for process timeline (instead of starting/stopping the watch, as every event including start/stop of process will include timestamp, we use event flagging to indicate events in the process timeline)\n* stop process / deregister\n\n## Miscelaneous\n* mention export LINENO before calling hume\n\n## Watchdog\n\n`humewatchdog.py` can be used to monitor that the `humed` daemon is alive.\nIt checks the daemon's pidfile and optionally executes a command when the\nprocess disappears.  The script may run once or periodically using the\n``--interval`` option:\n\n```\n$ python3 humewatchdog.py --interval 60 --alert-cmd 'systemctl restart humed'\n```\n\nWhen ``--interval`` is provided the watchdog will continue checking every\nspecified number of seconds.\n\n## TODO\n* Make humeconfig's --from-url work\n* add support to managing issues from within hume, example:\n\n```\n~$ humeconfig --update --github $GITHUB_TOKEN\nGithub Token configured.\nConfig file updated.\n\n~$ hume --beer --deliver-to=buanzo\nhume: No such command\n\n~$ hume --task hume --level critical --open-issue \"Hume support for beer brewing\"\nGithub Issue #45 created: github.com/hume/issues/45\n  \n[ some days later ]\n\n~$ hume --beer --deliver-to=buanzo --verbose\nOK\n~$ hume --task hume --level ok --fix-issue 45\nGithub Issue #45 fixed and closed\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuanzo%2Fhume","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuanzo%2Fhume","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuanzo%2Fhume/lists"}