{"id":13646416,"url":"https://github.com/skx/overseer","last_synced_at":"2025-04-23T03:51:07.715Z","repository":{"id":57554446,"uuid":"129231808","full_name":"skx/overseer","owner":"skx","description":"A golang-based remote protocol tester for testing sites \u0026 service availability","archived":false,"fork":false,"pushed_at":"2023-08-29T06:43:29.000Z","size":355,"stargazers_count":63,"open_issues_count":0,"forks_count":12,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-17T19:17:03.651Z","etag":null,"topics":["golang","monitoring","networking","testing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/skx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"skx","custom":"https://steve.fi/donate/"}},"created_at":"2018-04-12T10:06:00.000Z","updated_at":"2024-11-04T20:56:44.000Z","dependencies_parsed_at":"2024-01-14T09:59:34.843Z","dependency_job_id":"fa3dcafb-29dd-408f-a64a-63874b71dbde","html_url":"https://github.com/skx/overseer","commit_stats":null,"previous_names":[],"tags_count":23,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Foverseer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Foverseer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Foverseer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/skx%2Foverseer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/skx","download_url":"https://codeload.github.com/skx/overseer/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250366685,"owners_count":21418768,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","monitoring","networking","testing"],"created_at":"2024-08-02T01:02:55.085Z","updated_at":"2025-04-23T03:51:07.696Z","avatar_url":"https://github.com/skx.png","language":"Go","funding_links":["https://github.com/sponsors/skx","https://steve.fi/donate/"],"categories":["Go"],"sub_categories":[],"readme":"[![Go Report Card](https://goreportcard.com/badge/github.com/skx/overseer)](https://goreportcard.com/report/github.com/skx/overseer)\n[![license](https://img.shields.io/github/license/skx/overseer.svg)](https://github.com/skx/overseer/blob/master/LICENSE)\n[![Release](https://img.shields.io/github/release/skx/overseer.svg)](https://github.com/skx/overseer/releases/latest)\n\n\nTable of Contents\n=================\n\n* [Overseer](#overseer)\n* [Installation \u0026amp; Dependencies](#installation--dependencies)\n  * [Source Installation go \u0026lt;=  1.11](#source-installation-go---111)\n  * [Source installation go  \u0026gt;= 1.12](#source-installation-go---112)\n  * [Dependencies](#dependencies)\n* [Executing Tests](#executing-tests)\n  * [Running Automatically](#running-automatically)\n  * [Smoothing Test Failures](#smoothing-test-failures)\n* [Notifications](#notifications)\n* [Metrics](#metrics)\n* [Redis Specifics](#redis-specifics)\n* [Docker](#docker)\n* [Github Setup](#github-setup)\n\n\n# Overseer\n\nOverseer is a simple and scalable [golang](https://golang.org/)-based remote protocol tester, which allows you to monitor the state of your network, and the services running upon it.\n\n\"Remote Protocol Tester\" sounds a little vague, so to be more concrete this application lets you test that (remote) services are running, and has built-in support for performing testing against:\n\n* DNS-servers\n   * Test lookups of A, AAAA, MX, NS, and TXT records.\n* Finger\n* FTP\n* HTTP \u0026 HTTPS fetches.\n   * HTTP basic-authentication is supported.\n   * Requests may be DELETE, GET, HEAD, POST, PATCH, POST, \u0026 etc.\n   * SSL certificate validation and expiration warnings are supported.\n* IMAP \u0026 IMAPS\n* MySQL\n* NNTP\n* ping / ping6\n* POP3 \u0026 POP3S\n* Postgres\n* redis\n* rsync\n* SMTP\n* SSH\n* Telnet\n* VNC\n* XMPP\n\n(The implementation of the protocol-handlers can be found beneath the top-level [protocols/](protocols/) directory in this repository.)\n\nTests to be executed are defined in a simple text-based format which has the general form:\n\n     $TARGET must run $SERVICE [with $OPTION_NAME $VALUE] ..\n\nYou can see what the available tests look like in [the sample test-file](input.txt), and each of the included protocol-handlers are self-documenting which means you can view example usage via:\n\n     ~$ overseer examples [pattern]\n\nAll protocol-tests transparently support testing IPv4 and IPv6 targets, although you may globally disable either address family if you wish.\n\n\n\n## Installation \u0026 Dependencies\n\nThere are two ways to install this project from source, which depend on the version of the [go](https://golang.org/) version you're using.\n\nIf you just need the binaries you can find them upon the [project release page](https://github.com/skx/overseer/releases).\n\n\n### Source Installation go \u003c=  1.11\n\nIf you're using `go` before 1.11 then the following command should fetch/update `overseer`, and install it upon your system:\n\n     $ go get -u github.com/skx/overseer\n\n### Source installation go  \u003e= 1.12\n\nIf you're using a more recent version of `go` (which is _highly_ recommended), you need to clone to a directory which is not present upon your `GOPATH`:\n\n    git clone https://github.com/skx/overseer\n    cd overseer\n    go install\n\n\n### Dependencies\n\nBeyond the compile-time dependencies overseer requires a [redis](https://redis.io/) server which is used for two things:\n\n* As the storage-queue for parsed-jobs.\n* As the storage-queue for test-results.\n\nBecause overseer is executed in a distributed fashion tests are not executed\nas they are parsed/read, instead they are inserted into a redis-queue. A worker,\nor number of workers, poll the queue fetching \u0026 executing jobs as they become\navailable.\n\nIn small-scale deployments it is probably sufficient to have a single worker,\nand all the software running upon a single host.  For a larger number of\ntests (1000+) it might make more sense to have a pool of hosts each running\na worker.\n\nBecause we don't want to be tied to a specific notification-system results\nof each test are also posted to the same redis-host, which allows results to be retrieved and transmitted to your preferred notifier.\n\nMore details about [notifications](#notifications) are available later in this document.\n\n\n\n## Executing Tests\n\nAs mentioned already executing tests a two-step process:\n\n* First of all tests are parsed and inserted into a redis-based queue.\n* Secondly the tests are pulled from that queue and executed.\n\nThis might seem a little convoluted, however it is a great design if you\nhave a lot of tests to be executed, because it allows you to deploy multiple\nworkers.  Instead of having a single host executing all the tests you can\ncan have 10 hosts, each watching the same redis-queue pulling jobs, \u0026 executing\nthem as they become available.\n\nIn short using a central queue allows you to scale out the testing horizontally.\n\nTo add your tests to the queue you should run:\n\n       $ overseer enqueue \\\n           -redis-host=queue.example.com:6379 [-redis-pass='secret.here'] \\\n           test.file.1 test.file.2 .. test.file.N\n\nThis will parse the tests contained in the specified files, adding each of them to the (shared) redis queue.  Once all of the jobs have been parsed and inserted into the queue the process will terminate.\n\nTo drain the queue you can should now start a worker, which will fetch the tests and process them:\n\n       $ overseer worker -verbose \\\n          -redis-host=queue.example.com:6379 [-redis-pass='secret']\n\nThe worker will run constantly, not terminating unless manually killed.  With\nthe worker running you can add more jobs by re-running the `overseer enqueue`\ncommand.\n\nTo run tests in parallel simply launch more instances of the worker, on the same host, or on different hosts.\n\n\n\n### Running Automatically\n\nBeneath [systemd/](systemd/) you will find some sample service-files which can be used to deploy overseer upon a single host:\n\n* A service to start a single worker, fetching jobs from a redis server.\n  * The redis-server is assumed to be running on `localhost`.\n* A service \u0026 timer to regularly populate the queue with fresh jobs to be executed.\n  * i.e. The first service is the worker, this second one feeds the worker.\n\n\n\n### Smoothing Test Failures\n\nTo avoid triggering false alerts due to transient (network/host) failures\ntests which fail are retried several times before triggering a notification.\n\nThis _smoothing_ is designed to avoid raising an alert, which then clears\nupon the next overseer run, but the downside is that flapping services might\nnot necessarily become visible.\n\nIf you're absolutely certain that your connectivity is good, and that\nalerts should always be raised for failing services you can disable this\nretry-logic via the command-line flag `-retry=false`.\n\n\n\n## Notifications\n\nThe result of each test is submitted to the central redis-host, from where it can be pulled and used to notify a human of a problem.\n\nSample result-processors are [included](bridges/) in this repository which post\ntest-results to Telegram, a [purppura instance](https://github.com/skx/purppura), or via email.\n\nThe sample bridges are primarily included for demonstration purposes, the\nexpectation is you'll prefer to process the results and issue notifications to\nhumans via your favourite in-house tool - be it pagerduty, or something similar.\n\nThe results themselves are published as JSON objects to the `overseer.results` set.   Your notifier should remove the results from this set, as it generates alerts to prevent it from growing indefinitely.\n\nYou can check the size of the results set at any time via `redis-cli` like so:\n\n    $ redis-cli llen overseer.results\n    (integer) 0\n\nThe JSON object used to describe each test-result has the following fields:\n\n| Field Name | Field Value                                                     |\n| ---------- | --------------------------------------------------------------- |\n| `input`    | The input as read from the configuration-file.                  |\n| `result`   | Either `passed` or `failed`.                                    |\n| `error`    | If the test failed this will explain why.                       |\n| `time`     | The time the result was posted, in seconds past the epoch.      |\n| `target`   | The target of the test, either an IPv4 address or an IPv6 one.  |\n| `type`     | The type of test (ssh, ftp, etc).                               |\n\n**NOTE**: The `input` field will be updated to mask any password options which have been submitted with the tests.\n\nAs mentioned this repository contains some demonstration \"[bridges](bridges/)\", which poll the results from Redis, and forward them to more useful systems:\n\n* `email-bridge/main.go`\n  * This posts test-failures via email.\n  * Tests which pass are not reported.\n* `purppura-bridge/main.go`\n  * This forwards each test-result to a [purppura host](https://github.com/skx/purppura/).\n  * From there alerts will reach a human via pushover.\n* `telegram-bridge/main.go`\n  * This forwards each test-failure as a message to a Telegram user.\n\n\n\n## Metrics\n\nOverseer has built-in support for exporting metrics to a remote carbon-server:\n\n* Details of the system itself.\n   * Via the [go-metrics](https://github.com/skx/golang-metrics) package.\n* Details of the tests executed.\n   * Including the time to run tests, perform DNS lookups, and retry-counts.\n\nTo enable this support simply export the environmental variable `METRICS`\nwith the hostname of your remote metrics-host prior to launching the worker.\n\n\n\n## Redis Specifics\n\nWe use Redis as a queue as it is simple to deploy, stable, and well-known.\n\nRedis doesn't natively operate as a queue, so we replicate this via the \"list\"\nprimitives.  Adding a job to a queue is performed via a \"[rpush](https://redis.io/commands/rpush)\" operation, and pulling a job from the queue is achieved via an \"[blpop](https://redis.io/commands/blpop)\" command.\n\nWe use the following two lists as queues:\n\n* `overseer.jobs`\n    * For storing tests to be executed by a worker.\n* `overseer.results`\n    * For storing results, to be processed by a notifier.\n\nYou can examine the length of either queue via the [llen](https://redis.io/commands/llen) operation.\n\n* To view jobs pending execution:\n   * `redis-cli lrange overseer.jobs 0 -1`\n   * Or to view just the count\n      * `redis-cli llen overseer.jobs`\n* To view test-results which have yet to be notified:\n   * `redis-cli lrange overseer.results 0 -1`\n   * Or to view just the count\n      * `redis-cli llen overseer.results`\n\n\n\n\n## Docker\n\nThere are a series of Dockerfiles contained within this repository, they're designed to allow you to test things in a simple fashion.  However they do have the notification bridge hardcoded.\n\nYou can build the images like so:\n\n```\ndocker build -t overseer:bridge  -f Dockerfile.bridge .\ndocker build -t oversser:enqueue -f Dockerfile.enqueue .\ndocker build -t overseer:worker  -f Dockerfile.worker .\n```\n\nOnce built the supplied [docker-compose.yml](docker-compose.yml) file will let you launch them, using a shared redis instance.  The notifications will go via telegram by default, so you'll need to populate a token for a bot and setup your recipient user-ID.\n\n\n\n## Github Setup\n\nThis repository is configured to run tests upon every commit, and when pull-requests are created/updated.  The testing is carried out via [.github/run-tests.sh](.github/run-tests.sh) which is used by the [github-action-tester](https://github.com/skx/github-action-tester) action.\nReleases are automated in a similar fashion via [.github/build](.github/build), and the [github-action-publish-binaries](https://github.com/skx/github-action-publish-binaries) action.\n\nSteve\n--\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskx%2Foverseer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fskx%2Foverseer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fskx%2Foverseer/lists"}