{"id":13645981,"url":"https://github.com/hashicorp/consul-esm","last_synced_at":"2025-05-16T08:04:59.602Z","repository":{"id":28241434,"uuid":"107723650","full_name":"hashicorp/consul-esm","owner":"hashicorp","description":"External service monitoring for Consul","archived":false,"fork":false,"pushed_at":"2025-05-09T04:14:34.000Z","size":2284,"stargazers_count":262,"open_issues_count":54,"forks_count":41,"subscribers_count":37,"default_branch":"main","last_synced_at":"2025-05-09T04:27:39.586Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hashicorp.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2017-10-20T20:31:40.000Z","updated_at":"2025-05-09T03:49:36.000Z","dependencies_parsed_at":"2023-02-17T08:45:46.512Z","dependency_job_id":"4b0745ba-db21-4607-b4d8-487d97df0be6","html_url":"https://github.com/hashicorp/consul-esm","commit_stats":{"total_commits":314,"total_committers":44,"mean_commits":7.136363636363637,"dds":0.7006369426751593,"last_synced_commit":"af59e6c40b75403b8c06833bb4e6fe9e3e3d5627"},"previous_names":[],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashicorp%2Fconsul-esm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashicorp%2Fconsul-esm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashicorp%2Fconsul-esm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hashicorp%2Fconsul-esm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hashicorp","download_url":"https://codeload.github.com/hashicorp/consul-esm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254493378,"owners_count":22080126,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:02:46.219Z","updated_at":"2025-05-16T08:04:54.592Z","avatar_url":"https://github.com/hashicorp.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"[![Go Reference](https://pkg.go.dev/badge/github.com/hashicorp/consul-esm.svg)](https://pkg.go.dev/github.com/hashicorp/consul-esm)\n[![build](https://github.com/hashicorp/consul-esm/actions/workflows/build.yml/badge.svg)](https://github.com/hashicorp/consul-esm/actions/workflows/build.yml)\n[![ci](https://github.com/hashicorp/consul-esm/actions/workflows/ci.yml/badge.svg)](https://github.com/hashicorp/consul-esm/actions/workflows/ci.yml)\n\nConsul ESM (External Service Monitor)\n================\n\nThis project provides a daemon to run alongside Consul in order to run health\nchecks for external nodes and update the status of those health checks in the\ncatalog. It can also manage updating the coordinates of these external nodes,\nif enabled. See Consul's [External\nServices](https://learn.hashicorp.com/tutorials/consul/service-registration-external-services)\nguide for some more information about external nodes.\n\n## Community Support\n\nIf you have questions about how consul-esm works, its capabilities or\nanything other than a bug or feature request (use github's issue tracker for\nthose), please see our community support resources.\n\nCommunity portal: https://discuss.hashicorp.com/tags/c/consul/29/consul-esm\n\nOther resources: https://www.consul.io/community.html\n\nAdditionally, for issues and pull requests, we'll be using the :+1: reactions\nas a rough voting system to help gauge community priorities. So please add :+1:\nto any issue or pull request you'd like to see worked on. Thanks.\n\n## Prerequisites\n\nConsul ESM requires at least version 1.4.1 of Consul.\n\n| ESM version | Consul version required |\n|-------------|-------------------------|\n| 0.3.2 and higher | 1.4.1+ |\n| 0.3.1 and lower | 1.0.1-1.4.0 |\n\n## Installation\n\n1. Download a pre-compiled, released version from the [Consul ESM releases page][releases].\n\n1. Extract the binary using `unzip` or `tar`.\n\n1. Move the binary into `$PATH`.\n\nTo compile from source, please see the instructions in the\n[contributing section](#contributing).\n\n## Usage\n\nIn order for the ESM to detect external nodes and health checks, any external nodes must be registered\ndirectly with the catalog with `\"external-node\": \"true\"` set in the node metadata. Health checks can\nalso be registered with a 'Definition' field which includes the details of running the check. For example:\n\n```\n$ curl --request PUT --data @node.json localhost:8500/v1/catalog/register\n\nnode.json:\n\n{\n  \"Datacenter\": \"dc1\",\n  \"ID\": \"40e4a748-2192-161a-0510-9bf59fe950b5\",\n  \"Node\": \"foo\",\n  \"Address\": \"192.168.0.1\",\n  \"TaggedAddresses\": {\n    \"lan\": \"192.168.0.1\",\n    \"wan\": \"192.168.0.1\"\n  },\n  \"NodeMeta\": {\n    \"external-node\": \"true\",\n    \"external-probe\": \"true\"\n  },\n  \"Service\": {\n    \"ID\": \"web1\",\n    \"Service\": \"web\",\n    \"Tags\": [\n      \"v1\"\n    ],\n    \"Address\": \"127.0.0.1\",\n    \"Port\": 8000\n  },\n  \"Checks\": [{\n    \"Node\": \"foo\",\n    \"CheckID\": \"service:web1\",\n    \"Name\": \"Web HTTP check\",\n    \"Notes\": \"\",\n    \"Status\": \"passing\",\n    \"ServiceID\": \"web1\",\n    \"Definition\": {\n      \"HTTP\": \"http://localhost:8000/health\",\n      \"Interval\": \"10s\",\n      \"Timeout\": \"5s\"\n    }\n  },{\n    \"Node\": \"foo\",\n    \"CheckID\": \"service:web2\",\n    \"Name\": \"Web TCP check\",\n    \"Notes\": \"\",\n    \"Status\": \"passing\",\n    \"ServiceID\": \"web1\",\n    \"Definition\": {\n      \"TCP\": \"localhost:8000\",\n      \"Interval\": \"5s\",\n      \"Timeout\": \"1s\",\n      \"DeregisterCriticalServiceAfter\": \"30s\"\n     }\n  }]\n}\n```\n\nThe `external-probe` field determines whether the ESM will do regular pings to the node and\nmaintain an `externalNodeHealth` check for the node (similar to the `serfHealth` check used\nby Consul agents).\n\nThe ESM will perform a leader election by holding a lock in Consul, and the leader will then\ncontinually watch Consul for updates to the catalog and perform health checks defined on any\nexternal nodes it discovers. This allows externally registered services and checks to access\nthe same features as if they were registered locally on Consul agents.\n\nEach ESM registers a health check for itself with the agent with\n`\"DeregisterCriticalServiceAfter\": \"30m\"`, which is currently not configurable. This means after\nfailing its health check, the ESM will switch from passing status to critical status. If the ESM\nremains in critical status for 30 minutes, then the agent will attempt to deregister the ESM. During\ncritical status the ESM’s assigned external health checks will be reassigned to another ESM with\npassing status to monitor. Note: this is separate from the example JSON above for registering an\nexternal health check which has a `DeregisterCriticalServiceAfter` of 30 seconds.\n\n### Command Line\nTo run the daemon, pass the `-config-file` or `-config-dir` flag, giving the location of a config file\nor a directory containing .json or .hcl files.\n\n```\n$ consul-esm -config-file=/path/to/config.hcl -config-dir /etc/consul-esm.d\nConsul ESM running!\n            Datacenter: \"dc1\"\n               Service: \"consul-esm\"\n           Service Tag: \"\"\n            Service ID: \"consul-esm:5a6411b3-1c41-f272-b719-99b4f958fa97\"\nNode Reconnect Timeout: \"72h\"\n\nLog data will now stream in as it occurs:\n\n2017/10/31 21:59:41 [INFO] Waiting to obtain leadership...\n2017/10/31 21:59:41 [INFO] Obtained leadership\n2017/10/31 21:59:42 [DEBUG] agent: Check 'foo/service:web1' is passing\n2017/10/31 21:59:42 [DEBUG] agent: Check 'foo/service:web2' is passing\n```\n\n### Configuration\n\nConfiguration files can be provided in either JSON or [HashiCorp Configuration Language (HCL)][HCL] format.\nFor more information, please see the [HCL specification][HCL]. The following is an example HCL config file,\nwith the default values filled in:\n\n```hcl\n// The log level to use.\nlog_level = \"INFO\"\n\n// Controls whether to enable logging to syslog.\nenable_syslog = false\n\n// The syslog facility to use, if enabled.\nsyslog_facility = \"\"\n\n// Whether to log in json format\nlog_json = false\n\n// The unique id for this agent to use when registering itself with Consul.\n// If unconfigured, a UUID will be generated for the instance id.\n// Note: do not reuse the same instance id value for other agents. This id\n// must be unique to disambiguate different instances on the same host.\n// Failure to maintain uniqueness will result in an already-exists error.\ninstance_id = \"\"\n\n// The service name for this agent to use when registering itself with Consul.\nconsul_service = \"consul-esm\"\n\n// The service tag for this agent to use when registering itself with Consul.\n// ESM instances that share a service name/tag combination will have the work\n// of running health checks and pings for any external nodes in the catalog\n// divided evenly amongst themselves.\nconsul_service_tag = \"\"\n\n// The directory in the Consul KV store to use for storing runtime data.\nconsul_kv_path = \"consul-esm/\"\n\n// The node metadata values used for the ESM to qualify a node in the catalog\n// as an \"external node\".\nexternal_node_meta {\n    \"external-node\" = \"true\"\n}\n\n// The length of time to wait before reaping an external node due to failed\n// pings.\nnode_reconnect_timeout = \"72h\"\n\n// The interval to ping and update coordinates for external nodes that have\n// 'external-probe' set to true. By default, ESM will attempt to ping and\n// update the coordinates for all nodes it is watching every 10 seconds.\nnode_probe_interval = \"10s\"\n\n// Controls whether or not to disable calculating and updating node coordinates\n// when doing the node probe. Defaults to false i.e. coordinate updates\n// are enabled.\ndisable_coordinate_updates = false\n\n// The address of the local Consul agent. Can also be provided through the\n// CONSUL_HTTP_ADDR environment variable.\nhttp_addr = \"localhost:8500\"\n\n// The ACL token to use when communicating with the local Consul agent. Can\n// also be provided through the CONSUL_HTTP_TOKEN environment variable.\ntoken = \"\"\n\n// The Consul datacenter to use.\ndatacenter = \"dc1\"\n\n// The target Admin Partition to use.\npartition = \"\"\n\n// The CA file to use for talking to Consul over TLS. Can also be provided\n// though the CONSUL_CACERT environment variable.\nca_file = \"\"\n\n// The path to a directory of CA certs to use for talking to Consul over TLS.\n// Can also be provided through the CONSUL_CAPATH environment variable.\nca_path = \"\"\n\n// The client cert file to use for talking to Consul over TLS. Can also be\n// provided through the CONSUL_CLIENT_CERT environment variable.\ncert_file = \"\"\n\n// The client key file to use for talking to Consul over TLS. Can also be\n// provided through the CONSUL_CLIENT_KEY environment variable.\nkey_file = \"\"\n\n// The server name to use as the SNI host when connecting to Consul via TLS.\n// Can also be provided through the CONSUL_TLS_SERVER_NAME environment\n// variable.\ntls_server_name = \"\"\n\n// The CA file to use for talking to HTTPS checks.\nhttps_ca_file = \"\"\n\n// The path to a directory of CA certs to use for talking to HTTPS checks.\nhttps_ca_path = \"\"\n\n// The client cert file to use for talking to HTTPS checks.\nhttps_cert_file = \"\"\n\n// The client key file to use for talking to HTTPS checks.\nhttps_key_file = \"\"\n\n// Client address to expose API endpoints. Required in order to expose /metrics endpoint for Prometheus. Example: \"127.0.0.1:8080\"\nclient_address = \"\"\n\n// The method to use for pinging external nodes. Defaults to \"udp\" but can\n// also be set to \"socket\" to use ICMP (which requires root privileges).\nping_type = \"udp\"\n\n// The telemetry configuration which matches Consul's telemetry config options.\n// See Consul's documentation https://www.consul.io/docs/agent/options#telemetry\n// for more details on how to configure\ntelemetry {\n\tcirconus_api_app = \"\"\n \tcirconus_api_token = \"\"\n \tcirconus_api_url = \"\"\n \tcirconus_broker_id = \"\"\n \tcirconus_broker_select_tag = \"\"\n \tcirconus_check_display_name = \"\"\n \tcirconus_check_force_metric_activation = \"\"\n \tcirconus_check_id = \"\"\n \tcirconus_check_instance_id = \"\"\n \tcirconus_check_search_tag = \"\"\n \tcirconus_check_tags = \"\"\n \tcirconus_submission_interval = \"\"\n \tcirconus_submission_url = \"\"\n \tdisable_hostname = false\n \tdogstatsd_addr = \"\"\n \tdogstatsd_tags = []\n \tfilter_default = false\n \tprefix_filter = []\n \tmetrics_prefix = \"\"\n \tprometheus_retention_time = \"0\"\n \tstatsd_address = \"\"\n \tstatsite_address = \"\"\n}\n\n// The number of additional successful checks needed to trigger a status update to\n// passing. Defaults to 0, meaning the status will update to passing on the\n// first successful check.\npassing_threshold = 0\n\n// The number of additional failed checks needed to trigger a status update to\n// critical. Defaults to 0, meaning the status will update to critical on the\n// first failed check.\ncritical_threshold = 0\n```\n\n[HCL]: https://github.com/hashicorp/hcl \"HashiCorp Configuration Language (HCL)\"\n\n### Threshold for Updating Check Status\n\nTo prevent flapping, thresholds for updating a check status can be configured by `passing_threshold`\nand `critical_threshold` such that a check will update and switch to be passing / critical after an\nadditional number of consecutive or non-consecutive checks.\n\nBy default, these configurations are set to 0, which retains the original ESM behavior. If the status\nof a check is 'passing', then the next failed check will cause the status to update to be 'critical'.\nHence, the first failed check causes the update and 0 additional checks are needed.\n\nIf a check is currently 'passing' and configuration is `critical_threshold=3`, then after the first\nfailure, 3 additional consecutive failures (4 in total) are needed in order to update the status to 'critical'.\n\nESM also employs a counting system that allows for non-consecutive checks to aggregate and update\nthe check status. This counting system increments when a check result is the opposite of the current status\nand decrements when same as the current status.\n\nFor an example of how non-consecutive checks are counted, we have a check that has the status\n'passing', `critical_threshold=3`, and the counter is at 0 (c=0). The following pattern of pass/fail\nwill decrement/increment the counter as such:\n\n`PASS (c=0), FAIL (c=1), FAIL (c=2), PASS (c=1), FAIL (c=2), FAIL (c=3), PASS (c=2), FAIL (c=3), FAIL (c=4)`\n\nWhen the counter reaches 4 (1 initial fail + 3 additional fails), the critical_threshold is met and\nthe check status will update to 'critical' and the counter will reset.\n\nNote: this implementation diverges from [Consul's anti-flapping thresholds][Consul Anti-Flapping], which\ncounts total consecutive checks.\n\n[Consul Anti-Flapping]: https://www.consul.io/docs/agent/checks#success-failures-before-passing-warning-critical \"Consul Agent Success/Failures before passing/warning/critical\"\n\n### Consul ACL Policies\n\nWith [ACL system][ACL] enabled on Consul agents, a specific ACL policy may be\nrequired for ESM's token in order for ESM to perform its functions. To narrow\ndown the privileges required for ESM the following [ACL policy rules][rules]\ncan be used:\n\n```hcl\nagent_prefix \"\" {\n  policy = \"read\"\n}\n\nkey_prefix \"consul-esm/\" {\n  policy = \"write\"\n}\n\nnode_prefix \"\" {\n  policy = \"write\"\n}\n\nservice_prefix \"\" {\n  policy = \"write\"\n}\n\nsession_prefix \"\" {\n   policy = \"write\"\n}\n```\n\nThe `key_prefix` rule is set to allow the `consul-esm/` KV prefix, which is\ndefined in the config file using the `consul_kv_path` parameter.\n\n[ACL]: https://www.consul.io/docs/acl/acl-system.html \"Consul ACL System\"\n[rules]: https://www.consul.io/docs/acl/acl-rules \"Consul ACL Rules\"\n\nIt is possible to have even finer-grained ACL policies if you know the\nthe set name of the consul agent that ESM is registered with and a set list of\nnodes that ESM will monitor.\n - `\u003cconsul-agent-node-name\u003e`: insert the node name for the consul agent that\nconsul-esm is registered with\n - `\u003cmonitored-node-name\u003e`: insert the name of the nodes that ESM will\n monitor\n - `\u003cconsul-esm-name\u003e`: insert the name that ESM is registered with. Default\n value is 'consul-esm' if not defined in config file using the `consul_service`\n parameter\n\n```hcl\nagent \"\u003cconsul-agent-node-name\u003e\" {\n  policy = \"read\"\n}\n\nkey_prefix \"consul-esm/\" {\n  policy = \"write\"\n}\n\nnode \"\u003cmonitored-node-name: one acl block needed per node\u003e\" {\n  policy = \"write\"\n}\n\nnode_prefix \"\" {\n  policy = \"read\"\n}\n\nservice \"\u003cconsul-esm-name\u003e\" {\n  policy = \"write\"\n}\n\nsession \"\u003cconsul-agent-node-name\u003e\" {\n   policy = \"write\"\n}\n```\n\nFor context on usage of each ACL:\n\n- `agent:read` - for features to check version compatibility and calculating network coordinates\n- `key:write` - to store assigned checks\n- `node:write` - to update the status of each node that esm monitors\n- `node:read` - to retrieve nodes that need to be monitored\n- `service:write` - to register esm service\n- `session:write` - to acquire esm cluster leader lock\n\n### Consul Namespaces (Enterprise Feature)\n\nESM supports [Consul Enterprise Namespaces\n](https://www.consul.io/docs/enterprise/namespaces). When run with enterprise\nConsul servers it will scan all accessible Namespaces for external nodes and\nhealth checks to monitor. What is meant by \"all accessible\" is all Namespaces\naccessible via [Namespace ACL rules](\nhttps://www.consul.io/docs/security/acl/acl-rules) that provide `read` level\naccess to the Namespace. The simplest case of wanting to access all Namespaces\nwould add the below rule to the ESM ACL policy in the previous section...\n\n```hcl\nnamespace_prefix \"\" {\n  acl = \"read\"\n}\n```\n\nIf an ESM instance needs to monitor only a subset of existing Namespaces, the\npolicy will need to grant access to each Namespace explicitly. For example lets\nsay we have 3 Namespaces, \"foo\", \"bar\" and \"zed\" and you want this ESM to only\nmonitor \"foo\" and \"bar\". Your policy would need to have these listed (or a\ncommon prefix would work)...\n\n```hcl\nnamespace \"foo\" {\n  acl = \"read\"\n}\nnamespace \"bar\" {\n  acl = \"read\"\n}\n```\n\n#### Namespaces + `consul_kv_path` config setting:\n\n* If you have multiple ESMs for HA (secondary, backup ESMs) have the **same**\n  value set to `consul_kv_path`. (in practice these configs are identical)\n\n* If you have multiple ESMs for separate Namespaces each must use a\n  **different** setting for `consul_kv_path`.\n\nESM uses the `consul_kv_path` to determine where to keep its meta data. This\nmeta data will be different for each ESM monitoring different Namespaces.\n\nNote you can have both, those in HA clusters would have the same value and each\nseparate HA cluster would use different values.\n\n## Contributing\n\n**Note** if you run Linux and see `socket: permission denied` errors with UDP\nping, you probably need to modify system permissions to allow for non-root\naccess to the ports. Running `sudo sysctl -w net.ipv4.ping_group_range=\"0\n65535\"` should fix the problem (until you reboot, see sysctl man page for how\nto persist).\n\nTo build and install Consul ESM locally, you will need to install the\nDocker engine:\n\n- [Docker for Mac](https://docs.docker.com/engine/installation/mac/)\n- [Docker for Windows](https://docs.docker.com/engine/installation/windows/)\n- [Docker for Linux](https://docs.docker.com/engine/installation/linux/ubuntulinux/)\n\nClone the repository:\n\n```shell\n$ git clone https://github.com/hashicorp/consul-esm.git\n```\n\nTo compile the `consul-esm` binary for your local machine:\n\n```shell\n$ make dev\n```\n\nThis will compile the `consul-esm` binary into `bin/consul-esm` as\nwell as your `$GOPATH` and run the test suite.\n\nIf you want to compile a specific binary, run `make XC_OS/XC_ARCH`.\nFor example:\n```\nmake darwin/amd64\n```\n\nOr run the following to generate all binaries:\n\n```shell\n$ make build\n```\n\nIf you just want to run the tests:\n\n```shell\n$ make test\n```\n\nOr to run a specific test in the suite:\n\n```shell\ngo test ./... -run SomeTestFunction_name\n```\n\n[releases]: https://releases.hashicorp.com/consul-esm \"Consul ESM Releases\"\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhashicorp%2Fconsul-esm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhashicorp%2Fconsul-esm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhashicorp%2Fconsul-esm/lists"}