{"id":13542546,"url":"https://github.com/cloud66-oss/watchman","last_synced_at":"2025-04-10T13:31:36.881Z","repository":{"id":49585820,"uuid":"297460689","full_name":"cloud66-oss/watchman","owner":"cloud66-oss","description":"Simple service to check URL endpoints","archived":false,"fork":false,"pushed_at":"2021-06-19T08:32:40.000Z","size":70,"stargazers_count":20,"open_issues_count":1,"forks_count":1,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-24T21:42:29.558Z","etag":null,"topics":["golang","serverless","sitechecker"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloud66-oss.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-09-21T21:03:25.000Z","updated_at":"2024-05-25T14:49:03.000Z","dependencies_parsed_at":"2022-09-01T00:21:11.101Z","dependency_job_id":null,"html_url":"https://github.com/cloud66-oss/watchman","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud66-oss%2Fwatchman","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud66-oss%2Fwatchman/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud66-oss%2Fwatchman/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloud66-oss%2Fwatchman/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloud66-oss","download_url":"https://codeload.github.com/cloud66-oss/watchman/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248225708,"owners_count":21068078,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["golang","serverless","sitechecker"],"created_at":"2024-08-01T10:01:10.267Z","updated_at":"2025-04-10T13:31:31.872Z","avatar_url":"https://github.com/cloud66-oss.png","language":"Go","readme":"\u003cimg src=\"http://cdn2-cloud66-com.s3.amazonaws.com/images/oss-sponsorship.png\" width=150/\u003e\n\n# Watchman\nWatchman is a web service to check health of an HTTP(S) endpoint remotely. It is intendend to run as a light, multi-instance service on different geographical locations to provide a full picture of a web endpoint's health and accessibilty. \n\nWatchman can easily be deployed as a \"serverless\" function (like AWS Lambda or GCP Cloud Run) for full geographical distribution, access to high quality network and cost effective monitoring.\n\n## Usage\nWatchman is a single exectuable and can run on Linux, Mac, Windows or any other platform where you can compile it for (it's written in Go, so it's pretty portable). Once running, it will accept check requests and return the check results. \n\nTo use Watchman, first run it on your local machine: \n\n```bash\nwatchman\n```\n\nThis will start the process and listen to port 8080. Now you can use curl to send check requests to it:\n\n```bash\ncurl -H \"Content-Type: application/json\" http://localhost:8080 --data '{\"url\":\"http://www.google.com\",\"timeout\":\"1s\"}'\n```\n\n```json \n{\n  \"status\": 200,\n  \"error\": \"\",\n  \"dns_lookup\": 18228429,\n  \"tcp_connection\": 10254574,\n  \"tls_handshake\": 0,\n  \"server_processing\": 67631930,\n  \"content_transfer\": 561552,\n  \"name_lookup\": 18228429,\n  \"connect\": 28456431,\n  \"pre_transfer\": 0,\n  \"start_transfer\": 96114933,\n  \"total\": 96676485\n}\n```\n\nThe command above, runs a check, from your local machine, to `http://www.google.com` with a 1 second timeout. If `http://www.google.com` responds in less than 1 second, Watchman will return a JSON payload with the site's response metrics. The numbers are in nanoseconds.\n\nIf there is an error during the check (the site name is invalid or the site is down), the `status` will be `0` and the `error` will have more information on the error. For a successful check, you should always look for `200` in `status`.\n\n`tls_handshake` and `pre_transfer` will only be populated for `https` requests.\n\n### Request\nChecks can be requested with the following `POST` payload to Watchman:\n\n```json\n{\n    \"url\": \"https://www.google.com\",\n    \"timeout\": \"100ms\",\n    \"redirects_to_follow\": 3,\n    \"verify_certs\": true\n}\n```\n\nOnly `url` is required. The rest of the attributes are defaulted to the values above.\n\n### Response\nWatchman returns the following payload:\n\n```json\n{\n    \"status\": 200,\n    \"error\": \"\",\n    \"dns_lookup\": 18228429,\n    \"tcp_connection\": 10254574,\n    \"tls_handshake\": 0,\n    \"server_processing\": 67631930,\n    \"content_transfer\": 561552,\n    \"name_lookup\": 18228429,\n    \"connect\": 28456431,\n    \"pre_transfer\": 0,\n    \"start_transfer\": 96114933,\n    \"total\": 96676485\n}\n```\n\nWhen a check fails due to the site issues, `status` will be `0` and `error` will have more info on the error. The rest of the values will be `0`. \n\nA successful check will have `200` as `status` and an empty `error`. \n\nAn internal error in Watchman will return a `500` status code and the text of the error.\n\n## Configuration\nWatchman is configured using environment variables. This makes it suitable for serverless environments (see below). The following environment variables are supported:\n\n- `PORT`  Port to listen to (default 8080)\n- `TIMEOUT` Default check timeout. (default 100ms)\n- `MAX_REDIRECTS` Maximum number of HTTP redirects Watchman should follow before giving up (default 0: no redirect)\n- `AUTH_TOKEN`  If provided, Watchman will only respond to requests that have the same value in their X-Token header. (default none)\n- `SENTRY_API` If provided, Watchman will send crash reports to Sentry.\n- `_DEPLOY_REGION` Is used as an http header (`X-Region`) when calling the checked endpoint.\n\n## Deployment\nWatchman comes with a Dockerfile, so you can run it as a Docker container (ie on Kubernetes) or as a serverless deployment. Simply provide the required environment variables and start the executable. In production environment, make sure you secure the setup with SSL.\n\nYou can use the prebuilt image of Watchman:\n\n```bash\ndocker pull cloud66/watchman\n```\n\n\n### Deploying to GCP Cloud Run\nThis is a quick guide on how to deploy Watchman to multiple Google Cloud Run regions.\n\n\u003e Make sure your `gcloud` is configured and connected to your GCP account correctly.\n\n```bash\ngcloud builds submit --tag gcr.io/PROJECT-ID/watchman\n```\n\nThis will start the build process for the first time. Now you can deploy Watchman:\n\n\n```bash\ngcloud run deploy --image gcr.io/PROJECT-ID/watchman --platform managed\n```\n\n1. You will be prompted for the service name: press Enter to accept the default name, watchman.\n2. You will be prompted for region: select the region of your choice, for example us-central1.\n3. You will be prompted to allow unauthenticated invocations: respond y\n\nRepeat this process with as many regions as you'd like. Make sure to set the environment variables under the **Variables** section for the services in GCP.\n\n## Scheduling and Alerts\nRunning checks regularly and sending alerts when the endpoint is not reachable is not within the remit of Watchman, but it's easy to setup. You can use any hosted cloud scheduling services (like GCP Cloud Schedule) or your own cron jobs to do this by hitting Watchman endpoints and aggregating the restuls.\n","funding_links":[],"categories":["Go","serverless"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud66-oss%2Fwatchman","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloud66-oss%2Fwatchman","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloud66-oss%2Fwatchman/lists"}