{"id":30450255,"url":"https://github.com/deflect-ca/edgemanage","last_synced_at":"2025-08-23T13:26:24.788Z","repository":{"id":43470182,"uuid":"399519161","full_name":"deflect-ca/edgemanage","owner":"deflect-ca","description":"Managing the HTTP availability of a cluster of web servers via DNS","archived":false,"fork":false,"pushed_at":"2022-10-27T06:44:22.000Z","size":333,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":8,"default_branch":"main","last_synced_at":"2024-04-17T15:13:17.414Z","etag":null,"topics":["dns","python"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deflect-ca.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-08-24T15:37:01.000Z","updated_at":"2024-04-17T15:13:17.415Z","dependencies_parsed_at":"2023-01-20T07:34:13.782Z","dependency_job_id":null,"html_url":"https://github.com/deflect-ca/edgemanage","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/deflect-ca/edgemanage","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fedgemanage","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fedgemanage/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fedgemanage/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fedgemanage/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deflect-ca","download_url":"https://codeload.github.com/deflect-ca/edgemanage/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deflect-ca%2Fedgemanage/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271749048,"owners_count":24814115,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-23T02:00:09.327Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dns","python"],"created_at":"2025-08-23T13:25:58.765Z","updated_at":"2025-08-23T13:26:24.774Z","avatar_url":"https://github.com/deflect-ca.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"edgemanage3\n========\n\nEdgemanage is a tool for managing the HTTP availability of a cluster of\nweb servers via DNS. The machines tested are expected to be at risk of\nlarge volumes of traffic, attack or other potential instability. If a\nmachine is found to be underperforming, it is replaced by a more\nperformant host to ensure maximum availability.\n\nOverview\n--------\n\nEdgemanage is a simple script and Python library designed to be run at\nregular intervals, usually via crontab. The designed usecase was every\n60 seconds but larger figures can be used[^1].\n\nEdgemanage fetches an object from a lists of hosts over HTTP and uses\nthe time taken to retrieve the object to make decisions about which\nhosts are healthiest. These hosts are then written to a zone file as A\nrecords for the apex of a domain, in addition to inserting files\nstored in the zone includes directory. Simple checksumming of the\nlocal and remote objects also happens after fetching.\n\nThe zone files that Edgemanage writes are created via Jinja templates,\nwith SOA and NS data defined in the configuration file and the output\nformat being bind-compliant. The per-domain records that are included\nare plain ol' Bind style rules. Just don't include any SOA records.\n\nInstallation\n--------\nSee [INSTALL.md](https://github.com/equalitie/edgemanage/blob/master/INSTALL.md).\n\nOperation\n--------\n\nA host is considered to be in a healthy state (internally called\n\"pass\") when the object is returned under the `goodenough` value set\nin the configuration file. Hosts that return the fetched object under\nthe time specified will always be chosen first in case the need to\nreplace a host that is not in a healthy state.\n\nCare is taken to ensure that DNS changes are not made where they are\nnot needed - this means that if the last set of known healthy edges\nare in a passing state, there will be no change in DNS.\n\nEdgemanage maintains a store of historical fetches per host and can\nmake decisions based on this data. By default, if there are not enough\npassing hosts, Edgemanage will add hosts based on their average over a\ntime window, and failing that, their overall average.\n\nEdgemanage needs to be run regularly to be of use. I recommend running\nit via cron. If you're setting it up for the first time, I recommend\nrunning it in verbose mode (*-v*) and either dry run mode (*-n*) or\nwriting to a location that doesn't contain production information.\n\nEdgemanage maintains a statefile that is used for historical\ninformation about previous live hosts and last rotation times.\n\nIf a connection to a host is refused, the maximum time allowable will\nbe assigned to a host (thereby ensuring both its removal from the live\npool and also a backoff window via its averages).\n\nLogging/Output\n--------\n\nFor debugging, the use of the verbose mode is recommended. Using\nverbose mode disables logging to syslog.\n\nThe dry run mode will only read the statefile and log/print the\ndecisions that would be made (use of the verbose switch is\nrecommended).\n\nConfiguration\n--------\n\nThe \"object\" that edgemanage focuses could be absolutely anything - in\ntesting the file that was used was a simple text file. The only\nconcern is that an object that takes a long time runs the risk of\ncoming close to theoretical fetch times in slow situation, thereby\npotentially interrupting sequential runs. It's also worth noting that\nEdgemanage currently uses a simple requests\n[get](http://docs.python-requests.org/en/latest/api/#requests.get), so\ndownloading enormous objects will lead to memory issues. So eh, don't\ndo that.\n\nEdgemanage supports multiple \"networks\" - different groups of hosts to\nbe queried and used for writing zone files.\n\nEdgemanage uses the `dnschange_maxfreq` configuration option to limit\nthe number of rotations that can be undertaken in a certain time\nperiod. This is to limit churn that could lead to constantly empty\ncaches and so on.\n\nSee the `edgemanage.yaml` file for documentation of the configuration\noptions.\n\nCanaries\n-------\n\nSo-called \"canary\" edges are used to assign individual network\nresources to a single zone. They are a completely optional part of\nEdgemanage configuration, but may be useful for deploying special\nconfigurations, per-domain systems or for detection/analysis\napproaches.\n\nAn example of a use of this functionality would be if you had a number\nof systems that were present in a network environment where incoming\ntraffic is filtered upstream somehow. If canaries were to be included\nfor some domains with IP addresses corresponding to a system with\nunfiltered access, the canary IPs can be used for traffic capture and\nanalysis without needing to discard all other hosts. This approach is\nalso useful for low-cost setups where many sites are hosted and attack\ntraffic such as UDP is involved. Using a per-zone IP address allows\nfor a differential diagnosis of attack traffic, isolating which sites\nare attracting attacks.\n\nTo give a worked example - mydnet1 has a canary file in\n```/etc/edgemanage/canaries/mydnet1```. This path is set in\nedgemanage.yaml. On run, the file in\n```/etc/edgemanage/canaries/mydnet1``` is loaded and the YAML data is\nread (it should contain only a list of site: ipaddress pairs). Let's\nsay mydnet1 contains example.net: 10.0.2.22. Edgemanage\ntests 10.0.2.22 as it would any other edge but never selects it for\nwhat edgemanage considers to be \"liveness\". If\n10.0.2.22 is in a passing state, a random edge from the current live\nset is removed from example.net's configuration and 10.0.2.22 is\nadded. No other zones are affected and zone files are written as\nnormal.\n\nMonitoring\n--------\n\nStraight forward Nagios-compliant checks are available in the\n[nagios](nagios) directory. The checks are designed to use\nnothing but the Python standard library the files that the\n`edge_manage` script writes to the state and heath directories.\n\nHistory\n--------\n\nEdgemanage was developed as a replacement for a few aspects of the\n[Deflect](https://deflect.ca) project.\n\nThe name \"edgemanage\" is taken from the original Edgemanage tool in\nthe NodeJS [devopsjs](https://github.com/equalitie/devopsjs) toolset\nby David Mason. For various reasons, Edgemanage2 is written in Python.\n\n[^1]: Figures less than 60 seconds are actually outright forbidden as\nit somewhat negates the purpose of the tool. Dry run mode can be used\nto run more regularly with no file writing.\n\nPython 2 to 3 upgrading process start around July of 2020, as Python 2\nis no longer supported.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeflect-ca%2Fedgemanage","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeflect-ca%2Fedgemanage","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeflect-ca%2Fedgemanage/lists"}