Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cloud66-oss/watchman
Simple service to check URL endpoints
https://github.com/cloud66-oss/watchman
golang serverless sitechecker
Last synced: about 14 hours ago
JSON representation
Simple service to check URL endpoints
- Host: GitHub
- URL: https://github.com/cloud66-oss/watchman
- Owner: cloud66-oss
- License: apache-2.0
- Created: 2020-09-21T21:03:25.000Z (about 4 years ago)
- Default Branch: master
- Last Pushed: 2021-06-19T08:32:40.000Z (over 3 years ago)
- Last Synced: 2024-08-01T10:17:33.801Z (3 months ago)
- Topics: golang, serverless, sitechecker
- Language: Go
- Homepage:
- Size: 68.4 KB
- Stars: 20
- Watchers: 4
- Forks: 1
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome - cloud66-oss/watchman - Simple service to check URL endpoints (serverless)
README
# Watchman
Watchman is a web service to check health of an HTTP(S) endpoint remotely. It is intendend to run as a light, multi-instance service on different geographical locations to provide a full picture of a web endpoint's health and accessibilty.
Watchman can easily be deployed as a "serverless" function (like AWS Lambda or GCP Cloud Run) for full geographical distribution, access to high quality network and cost effective monitoring.
## Usage
Watchman is a single exectuable and can run on Linux, Mac, Windows or any other platform where you can compile it for (it's written in Go, so it's pretty portable). Once running, it will accept check requests and return the check results.
To use Watchman, first run it on your local machine:
```bash
watchman
```
This will start the process and listen to port 8080. Now you can use curl to send check requests to it:
```bash
curl -H "Content-Type: application/json" http://localhost:8080 --data '{"url":"http://www.google.com","timeout":"1s"}'
```
```json
{
"status": 200,
"error": "",
"dns_lookup": 18228429,
"tcp_connection": 10254574,
"tls_handshake": 0,
"server_processing": 67631930,
"content_transfer": 561552,
"name_lookup": 18228429,
"connect": 28456431,
"pre_transfer": 0,
"start_transfer": 96114933,
"total": 96676485
}
```
The command above, runs a check, from your local machine, to `http://www.google.com` with a 1 second timeout. If `http://www.google.com` responds in less than 1 second, Watchman will return a JSON payload with the site's response metrics. The numbers are in nanoseconds.
If there is an error during the check (the site name is invalid or the site is down), the `status` will be `0` and the `error` will have more information on the error. For a successful check, you should always look for `200` in `status`.
`tls_handshake` and `pre_transfer` will only be populated for `https` requests.
### Request
Checks can be requested with the following `POST` payload to Watchman:
```json
{
"url": "https://www.google.com",
"timeout": "100ms",
"redirects_to_follow": 3,
"verify_certs": true
}
```
Only `url` is required. The rest of the attributes are defaulted to the values above.
### Response
Watchman returns the following payload:
```json
{
"status": 200,
"error": "",
"dns_lookup": 18228429,
"tcp_connection": 10254574,
"tls_handshake": 0,
"server_processing": 67631930,
"content_transfer": 561552,
"name_lookup": 18228429,
"connect": 28456431,
"pre_transfer": 0,
"start_transfer": 96114933,
"total": 96676485
}
```
When a check fails due to the site issues, `status` will be `0` and `error` will have more info on the error. The rest of the values will be `0`.
A successful check will have `200` as `status` and an empty `error`.
An internal error in Watchman will return a `500` status code and the text of the error.
## Configuration
Watchman is configured using environment variables. This makes it suitable for serverless environments (see below). The following environment variables are supported:
- `PORT` Port to listen to (default 8080)
- `TIMEOUT` Default check timeout. (default 100ms)
- `MAX_REDIRECTS` Maximum number of HTTP redirects Watchman should follow before giving up (default 0: no redirect)
- `AUTH_TOKEN` If provided, Watchman will only respond to requests that have the same value in their X-Token header. (default none)
- `SENTRY_API` If provided, Watchman will send crash reports to Sentry.
- `_DEPLOY_REGION` Is used as an http header (`X-Region`) when calling the checked endpoint.
## Deployment
Watchman comes with a Dockerfile, so you can run it as a Docker container (ie on Kubernetes) or as a serverless deployment. Simply provide the required environment variables and start the executable. In production environment, make sure you secure the setup with SSL.
You can use the prebuilt image of Watchman:
```bash
docker pull cloud66/watchman
```
### Deploying to GCP Cloud Run
This is a quick guide on how to deploy Watchman to multiple Google Cloud Run regions.
> Make sure your `gcloud` is configured and connected to your GCP account correctly.
```bash
gcloud builds submit --tag gcr.io/PROJECT-ID/watchman
```
This will start the build process for the first time. Now you can deploy Watchman:
```bash
gcloud run deploy --image gcr.io/PROJECT-ID/watchman --platform managed
```
1. You will be prompted for the service name: press Enter to accept the default name, watchman.
2. You will be prompted for region: select the region of your choice, for example us-central1.
3. You will be prompted to allow unauthenticated invocations: respond y
Repeat this process with as many regions as you'd like. Make sure to set the environment variables under the **Variables** section for the services in GCP.
## Scheduling and Alerts
Running checks regularly and sending alerts when the endpoint is not reachable is not within the remit of Watchman, but it's easy to setup. You can use any hosted cloud scheduling services (like GCP Cloud Schedule) or your own cron jobs to do this by hitting Watchman endpoints and aggregating the restuls.