{"id":18579233,"url":"https://github.com/faust64/highwaytohell","last_synced_at":"2025-04-10T10:31:28.498Z","repository":{"id":76091505,"uuid":"99932546","full_name":"faust64/highwaytohell","owner":"faust64","description":"PoC for a Route53 alternative","archived":false,"fork":false,"pushed_at":"2024-04-18T16:38:33.000Z","size":58548,"stargazers_count":5,"open_issues_count":1,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-24T19:39:59.060Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/faust64.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-08-10T14:33:32.000Z","updated_at":"2023-01-21T14:01:27.000Z","dependencies_parsed_at":"2023-12-12T01:29:36.568Z","dependency_job_id":"e6d76503-8f07-411e-b4f7-8e7637606be6","html_url":"https://github.com/faust64/highwaytohell","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faust64%2Fhighwaytohell","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faust64%2Fhighwaytohell/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faust64%2Fhighwaytohell/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faust64%2Fhighwaytohell/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/faust64","download_url":"https://codeload.github.com/faust64/highwaytohell/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248199136,"owners_count":21063641,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-06T23:39:33.743Z","updated_at":"2025-04-10T10:31:27.010Z","avatar_url":"https://github.com/faust64.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HighWayToHell\n\n[![CircleCI build status](https://circleci.com/gh/faust64/highwaytohell/tree/master.svg?style=shield)](https://circleci.com/gh/faust64/highwaytohell/tree/master)\n\nTable of Contents\n=================\n\n  * [HighWayToHell](#highwaytohell)\n    * [Introducing HWTH](#introducing-hwth)\n      * [Pool](#pool)\n      * [What does it do](#what-does-it-do)\n      * [Scaling Out](#scaling-out)\n      * [Todolist](#todolist)\n    * [Databases](#databases)\n    * [Introducing Workers](#introducing-workers)\n      * [refreshZones](#refreshzones)\n      * [checkHealth](#checkhealth)\n      * [outboundNotifier](#outboundnotifier)\n      * [apiGW](#apigw)\n    * [Special Thanks](#special-thanks)\n\n## Introducing HWTH\n\n * Some hopefully-scalable, DNSSEC-capable, DNS manager featuring health checks\n   \u0026 alerts configuration (HTTP POST/GET, SMS or email).\n * See [QuickStart](QUICKSTART.md) for practical instructions building packages,\n   deploying your setup, ...\n * See [our API doc](API.md) for a list of routes and some thoughts and notes.\n * Any advices, contribution or feedback welcome.\n\n### Pool\n\n![HWTH Pool Illustrated](samples.d/diags/hwth.png)\n\n### What does it do\n\n * starting the `refreshZones` worker, you would be able to generate NSD or\n   BIND configurations \u0026 corresponding zones files. Note before doing so,\n   you would have to start a `hwth-watchmark` service in charge of reloading\n   your nameserver configuration - as root, while our NodeJS user can't.\n * starting the `checkHealth` worker, you would be able to run health checks\n   that may eventually be used as conditions generating your DNS zones or\n   scheduling notifications\n * starting the `apiGW` worker - and configuring some SSL-capable reverse\n   proxy forwarding connections to your lookpack on port 8080, you would\n   have access to a web client declaring domains, records, health checks,\n   creating API tokens for CLI usage (see `samples.d/butters`), enabling\n   2FA protection via authenticator such as Authy \u0026 optional OTP backup codes,\n   sharing your zones management with other users.\n * starting the `outboundNotifier` worker, you should be able to configure\n   POST/GET/email/SMS notifications based on your health check statuses,\n   as well as notifications on login and/or failed login accessing our\n   web service\n\n### Scaling out\n\n![Scaling out HWTH](samples.d/diags/hwth-distribution.png)\n\n### Todolist\n\n * handling redis authentication\n * paging (?)\n * zones import tool?\n * moar tests\n * api-less mode?\n * shinyness - CSS or frontend contributions most welcome\n * DNSSEC keys rotation open to discussion, bearing in mind it implies\n   publishing new DS records to registrar, we can't automate it unilaterally\n * reproducible benchmarks \u0026 gnuplot magic ...\n\n## Databases\n\nUsing a Redis backend - as a jobs queue, pub/sub, sessions storage,\n2FA-establishing-token \u0026 2FA-validated-token storage\n\nUsing a Cassandra backend storing pretty much everything else. The followings\ntables would be used:\n\n * users: account-specific settings\n * twofa: a collection of 2fa secrets, mapped to their owner\n * backupcodes: a collection of 2fa backup codes, mapped to their owner\n * tokens: a collection of tokens, mapped to their owner. TODO: a permissions\n   string is defined, yet not used\n * contactaddresses: collection of contact addresses (only emails so far,\n   could eventually include phone numbers) mapped to their owner\n * adding phone number as contact, shouldn't ask to check emails, rather sms\n * logins: a login history collection, associating an user ID to a client IP,\n   a timestamp and wether login succeeded or failed\n * nspools: inventory of ns pools\n * zones: zones inventory and global settings, mapped to their nspool\n * rbaclookalike: maps users to zones to a role\n * records: DNS records definitions, mapped to their zone\n * checks: health checks definitions, mapped to their zone\n * checkhistory: health checks history, mapped to a check\n * notifications: a collection of conditions and target to notify, when service\n   health changes, mapped to a check\n * dnsseckeys: storing base64-encoded ZSK \u0026 KSK keys, mapped to a ZSK \u0026 KSK\n   key names, as listed in the zones table\n * signedzones: storing base64-encoded DNSSEC zones, once they're signed, for\n   our neighbors to get their copy. mapped to a domain\n * config: a dummy table we would keep our schema verion into, so our database\n   upgrade script can identify which patches to skip or apply.\n\n## Introducing Workers\n\n### refreshZones\n\nA first class of worker is in charge of generating zones. `refreshZones`\nconnects to a couple of bull queue, and also opens a pair of\npublisher/subscriber to redis.\nWorkers from a pool receive refresh notifications (from our API gateway or\nhealth check workers) via the bull queues.\nUpon completion, we send the corresponding domain name into our pubsub, so\nthat our neighbors eventually reload their own zones as well.\n\nIf a zone is subject to DNSSEC, then a signed copy is uploaded to Cassandra\nwhen zone gets updated - and that copy gets installed to neighbors.\n\nNote the `refreshZones` worker, running from an unprivileged user, would not be\nable to reload your name servers. To address this, there is a second service you\nwould need to enable on any `refreshZones` worker that serves DNS zones. Said\nprocess would run as root, using `inotifywait` to reload `nsd` or `bind`,\nwhenever a mark file gets updated in the process of refreshing zones. See\n[QuickStart](QUICKSTART.md) for further details setting up your workers.\n\nFIXME:\n\n * resolving NSs in charge for a zone, we have a\n   `SELECT fqdn FROM nspools WHERE tag IN ('master', 'backup')`. It gives us the\n   pair of nameserver FQDNs to include generating a zone. Now note that when\n   your nspool tag name alphabetically succeeds your bkppool tag name, then\n   `SELECT` would return nameserver FQDNs such as your bkppool would actually be\n   considered to be your nspool, and vice versa.\n * ensure confQueue \u0026 zonesQueue are not applying some change simultaneously\n   (some kind of lock ...)\n\nDISCUSS:\n\n * do we need keeping plaintext zones when using DNSSEC?\n * we assume running name server on that worker, we could split it so a worker\n   generates (\u0026 signs) zones (without necessarily running a name server\n   locally), while an other one would only gets stuff we know passed checkzone\n   (\u0026 got signed) then actually restarting their name server. Note: the\n   generation/signature process does involve a checkzone that implies nsd or\n   bind utils were installed, regardless of being name servers.\n\n### checkHealth\n\nA second class of worker is in charge of running health checks. `checkHealth`\nsetups a few schedules.\nThe first one iterates over the health checks declared in Cassandra, running\nthose that need to be refreshed and adding records to our health checks history\ntable.\nA couple others purges older records from that history table, and checks that\nmight be referring to domains end-users would have purged.\n\n### outboundNotifier\n\nThis class of worker would be listening for events from our other workers,\neventually sending HTTP POST, GET, SMS or email notifications.\n\nThe `checkHealth` worker may schedule notification settings to be checked for\nmatching configurations, having refreshed a check status.\n\nThe `apiGW` worker may schedule login history to be checked notifying user\nhis account was accessed.\n\n### apiGW\n\nMinimalist API gateway (we've proven it can be done ... I don't necessarily\nenjoy customizing CSSs, be ensured I didn't test rendering on IExplorer ...),\nwith token authentication, 2FA-capable.\n\nFIXME:\n\n * dont res.send.(errorcode) if req.sessions.userid: instead render a common\n   template\n * error \u0026 confirmation pages back links \u0026 labels\n\n## Special Thanks\n\n * First and foremost [PeerioTechnologies](https://www.peerio.com), my current employer\n * As well as [Clement Duhart](https://github.com/slash6475), for introducing me to NodeJS\n * [StackOverflow](https://stackoverflow.com), answering my most obscure questions\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaust64%2Fhighwaytohell","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffaust64%2Fhighwaytohell","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaust64%2Fhighwaytohell/lists"}