{"id":19388319,"url":"https://github.com/mk-fg/name-based-routing-policy-controller","last_synced_at":"2026-06-17T11:32:18.587Z","repository":{"id":59113320,"uuid":"472290906","full_name":"mk-fg/name-based-routing-policy-controller","owner":"mk-fg","description":"Tool for monitoring service availibility and policy-routing around the issues","archived":false,"fork":false,"pushed_at":"2026-05-08T10:35:12.000Z","size":137,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-05-08T12:30:50.255Z","etag":null,"topics":["access","censorship-circumvention","filtering","network","policy","probe","python","routing","tunneling","web"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"wtfpl","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mk-fg.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-03-21T10:38:26.000Z","updated_at":"2026-05-08T10:35:16.000Z","dependencies_parsed_at":"2024-11-10T10:12:38.608Z","dependency_job_id":"e85b7269-0d05-44b1-9f99-9e6928d50fcf","html_url":"https://github.com/mk-fg/name-based-routing-policy-controller","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/mk-fg/name-based-routing-policy-controller","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fname-based-routing-policy-controller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fname-based-routing-policy-controller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fname-based-routing-policy-controller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fname-based-routing-policy-controller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mk-fg","download_url":"https://codeload.github.com/mk-fg/name-based-routing-policy-controller/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mk-fg%2Fname-based-routing-policy-controller/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34447264,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-17T02:00:05.408Z","response_time":127,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["access","censorship-circumvention","filtering","network","policy","probe","python","routing","tunneling","web"],"created_at":"2024-11-10T10:12:27.184Z","updated_at":"2026-06-17T11:32:18.581Z","avatar_url":"https://github.com/mk-fg.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Name-based Routing Policy Controller (nbrpc)\n\nA tool for monitoring service availability and policy-routing around\nthe issues deliberately and selectively created on either side, aka censorshit.\n\n\u003e \"The Net interprets censorship as damage and routes around it.\"\n\u003e -- John Gilmore\n\n\"The Net\" won't do it all by itself however, hence the tool here.\n\nEspecially useful these days, when local state, everyone around,\nand the rest of the world hate you, working together to cut you off\nfrom the interwebs, if you happen to live in a wrong place at a wrong time,\nwhich seem to be a global trend.\n\nTable of Contents\n\n- [More Description](#hdr-more_description)\n- [Routing policy decision-making logic](#hdr-routing_policy_decision-making_logic)\n- [Setup and usage](#hdr-setup_and_usage)\n- [Check list file format]\n- [Setup example with linux policy routing]\n- [Related links, tips, info and trivia]\n\n[Check list file format]: #hdr-check_list_file_format\n[Setup example with linux policy routing]:\n  #hdr-setup_example_with_linux_policy_routing\n[Related links, tips, info and trivia]:\n  #hdr-related_links__tips__info_and_trivia\n\nRepository URLs:\n\n- https://github.com/mk-fg/name-based-routing-policy-controller\n- https://codeberg.org/mk-fg/name-based-routing-policy-controller\n- https://fraggod.net/code/git/name-based-routing-policy-controller\n\n\n\u003ca name=hdr-more_description\u003e\u003c/a\u003e\n## More Description\n\nIt is a script for monitoring DNS names for whether services on their IPs\nare not accessible through direct connections and configuring linux (probably)\npolicy routing to use alternative paths to those hosts.\n\nPurpose is to work around access issues to often-used known-in-advance\nwebsites/services that get blocked on either side of the route to them\n(by either state censorship or geo-blocking), without running all traffic\nthrough tunnels unnecessarily, and without manually configuring proxies/tunnels\nin specific apps.\n\nList of handled hostnames is supposed to be maintained manually,\nwith script issuing notifications when those don't need to be on the list anymore,\ni.e. when direct route to those works, which requires script itself to be excluded\nfrom the routing policy that it sets up.\n\nActual routing configuration should be changed by other hook scripts, likely\nrunning with more access to update [nftables sets], [ip-route tables],\n[ip-rules] or somesuch - see examples below for more info.\n\nIt'd also make sense to use reasonably secure/private DNS resolution\nalongside this tool - see [DNS Privacy project] for more info on that.\n\nScripts here are not intended to do any tricks to fool DPI, discard RST\npackets or otherwise work around specific censorshit types and implementations,\njust check and configure workaround-routes via existing tunnels/interfaces.\n\nFor establishing tunnels or obfuscating traffic in other ways (bypassing\nspecific national firewall quirks), lookup tools like [XTLS Xray/REALITY],\n[trojan], [hysteria], [olcrtc], [gecit], [GoodbyeDPI], [NaïveProxy],\n[zapret2] instead - whichever work under your censorship regime, or resources\nlike [ntc.party] and [net4people/bbs] for more info on what might still work.\n\nThis script also will not work for hiding or obfuscating the fact of\naccessing something (which is a very complex and different subject),\nI'd recommend looking at stuff like [Tor Project] and similar privacy\ntoolkits/orgs, wrapping all traffic indiscriminately (not selectively)\nfrom an anonymized and security/privacy-hardened OS/VM/endpoint.\n\n[nftables sets]: https://wiki.nftables.org/wiki-nftables/index.php/Sets\n[ip-route tables]: https://man.archlinux.org/man/ip-route.8.en\n[ip-rules]: https://man.archlinux.org/man/ip-rule.8.en\n[DNS Privacy project]: https://dnsprivacy.org/\n[XTLS Xray/REALITY]: https://github.com/XTLS\n[trojan]: https://github.com/trojan-gfw\n[hysteria]: https://github.com/apernet/hysteria\n[olcrtc]: https://github.com/openlibrecommunity/olcrtc\n[gecit]: https://github.com/boratanrikulu/gecit\n[GoodbyeDPI]: https://github.com/ValdikSS/GoodbyeDPI\n[NaïveProxy]: https://github.com/klzgrad/naiveproxy\n[zapret2]: https://github.com/bol-van/zapret2\n[ntc.party]: https://ntc.party/\n[net4people/bbs]: https://github.com/net4people/bbs/issues\n[Tor Project]: https://www.torproject.org/\n\n\n\u003ca name=hdr-routing_policy_decision-making_logic\u003e\u003c/a\u003e\n## Routing policy decision-making logic\n\nGeneral idea and some less-obvious quirks of availability-checking\ndone by the script are listed below.\n\n\n- Whole tool can be decomposed into following parts:\n\n    - Local sqlite db file, storing all service availability and checks info.\n\n    - Service availability checking backend.\n\n        Selects/rate-limits/schedules which checks to run and runs them,\n        where each check can be split in two parts:\n\n        - DNS resolution to an ever-changing list of cloud IPs.\n\n        - Probing individual DNS name + IPv4/IPv6 address combinations via curl or such.\n\n    - Actions on availability changes.\n\n        - Applying re-routing workarounds \"policy\", via separate script,\n          which might need additional privileges to e.g. manipulate host's nftables.\n\n        - Applying DNS workarounds/overrides.\n\n    - Separate glue script(s) updating routing policies\n      or DNS zones, called from actions above.\n\n    It's useful to think about these concepts/components separately like that,\n    to talk about them in relative isolation, as is mostly done below.\n\n\n- DNS parts can be tricky.\n\n    Service DNS names are expected to resolve to multiple IPs, which change anytime,\n    as CDNs hand them out randomly depending on time, place, load, phase of the\n    moon or whatever.\n\n    This introduces a problem that apps can use different IP from checked ones,\n    unless either some DNS hack is introduced, or checks are run sufficiently often\n    and remember/re-route old IPs if those get handed-out in a round-robin fashion.\n\n    So, DNS resolution for a particular site/service can be handled in following ways:\n\n    - Not altered, if its IPs are stable enough, they're checked often enough\n      and/or with enough history, or just hitting occasional unchecked/blocked\n      one is not a big deal.\n\n    - Limit service to checked/rerouted IPs.\n\n        Safe opposite to above, to have apps use maybe slightly-stale addrs that\n        were confirmed to work, one way or another.\n\n    - Limit service to IPs that were checked and confirmed to be working directly,\n      maybe even older ones from earlier queries (if host rotates returned IPs).\n\n        Useful for high-traffic or latency-sensitive services, where\n        poorly-implemented censorshit laws block random IPs from the pool,\n        resulting in stuff occasionally timing-out at random, but tunneling it\n        can be less convenient/slow/costly than just dropping these unlucky addrs.\n\n    When some form of DNS override/filtering is in place, script can be used\n    with `-Z/--zone-for` option to export records for that at any time,\n    selecting strategies from the list above on per-host or per-run basis.\n\n    Option dumps local-zone info to stdout (in [Unbound] resolver format by default),\n    filtered by regexp for hostname(s) and any policy modifiers.\n    Using larger superset of \"all seen\" addresses can be useful to schedule\n    these updates less often, and not bother tracking upstream results exactly.\n\n\n- Enabling workarounds on failed connection checks can be done in different ways too.\n\n    - Reroute all old-and-current service IPs\n      if any/some/all of them are confirmed to be blocked.\n\n        It's useful to have longer grace periods or run alt-route checker here,\n        to avoid flapping workarounds on/off whenever services have any common\n        temporary issues on any of their endpoints.\n\n    - Same as above, but re-route specific address family (IPv4/IPv6),\n      when IPs in there are detected to be inaccessible.\n\n        This takes into account the fact that censorship is often simple,\n        and applies to a list of some IPv4 ranges only, as well as the fact\n        that IPv6 often gets broken on its own, so it's useful to treat these\n        AFs and their specific issues separately.\n\n        Idea is kinda similar to [Happy Eyeballs algorithm], which is widely used\n        when establishing connections with both IPv4/IPv6 options available.\n\n    - Reroute/tunnel only blocked-somewhere IPs that don't pass the checks.\n\n        Can be a smart way to do it with larger CDNs or an even dumber censorshit.\n\n    - Forego routing workarounds entirely in favor of some other solution.\n      DNS workarounds (filtering-out blocked addrs) or notifications\n      for something manual, for example.\n\n    These strategies can be toggled via global `-p/--check-list-default-policy`\n    option and set on a per-service/host basis to handle different things differently.\n\n    For small or known-blocked sites it can be easier to have broad \"reroute it all\"\n    policies, but might not be worth clogging the tunnel with all cloudflare, youtube\n    or twitch.tv video traffic at all, and only work around issues there on the DNS level,\n    if possible.\n\n\n- Checking \"hostname + address\" combination tends to be special for each host.\n\n    Default checks (\"https\") are not just ICMP pings or TCP connections,\n    but a curl page fetch, expecting specific http response codes,\n    to catch whatever mid-https RST packets (often for downgrade to ISP's http\n    blacklist page) and hijacking with bogus certs, which seem to be common for\n    censorship-type filtering situation.\n\n    It's useful to check and customize which response code is expected by using\n    e.g. \"api.twitter.com=404\" or query specific URL paths that return specific\n    http results, e.g. \"somesite.com:https/api/v5=400\", especially if generic\n    redirect responses are known to indicate access failure (leading to either\n    censorshit or a F-U page).\n\n\n- Good service availability check for specific address consists of two parts -\n  checking it via direct connection, and checking it via alternate route that's\n  supposed to be used as a workaround.\n\n    This is done so that checks don't just track general upstream up/down status,\n    but only mark things as needing workaround when it legitimately works that way,\n    unlike direct connection.\n\n\n- State of hosts in db only gets changed after a grace period(s), to avoid\n  flapping between routes needlessly during whatever temporary issues, like\n  maybe service being down in one geo-region or on some frontend IPs for a bit.\n\n    Both directions have different timeouts and transition rules - e.g. flipping\n    to workaround state is faster than back to direct connections by default,\n    and is done through intermediate \"failing\" state, with possible alt-route\n    checks in-between, to stall the transition if endpoint seem to be down from\n    both network perspectives.\n\n    All timeouts, intervals and delays are listed in `-h/--help` output and are\n    easily configurable.\n\n\n- Non-global/public addrs (as in iana-ipv4/ipv6-special-registry) are ignored in\n  getaddrinfo() results for all intents and purposes, to avoid hosts assigning\n  junk IPs messing with any checks or local routing.\n\n[Happy Eyeballs algorithm]: https://datatracker.ietf.org/doc/html/rfc6555\n\n\n\u003ca name=hdr-setup_and_usage\u003e\u003c/a\u003e\n## Setup and usage\n\nMain [nbrpc.py] is just one Python (3.9+) script that only needs common [curl]\ntool for its http(s) checks.\nGrab and drop it into any path, run with `-h/--help` option to get started.\n`--debug` option there can be used to get more insight into what script is doing.\n\nMain script runs availability checks, but doesn't do anything beyond that by default.\n\nIt expects a list of services/endpoints to check with `-f/--check-list-file`\noption, format for which is documented in [Check list file format] section below.\n\nHook scripts/commands can be run directly with `--policy-*-cmd` options,\nto control whatever system used for connection workarounds, or send this data\nto unix socket (`-s/--policy-socket` option), e.g. to something more privileged\noutside its sandbox that can tweak the firewall.\n\n[nbrpc-policy-cmd.py] and [nbrpc-policy-nft.py] scripts in the repo can be used\ninstead of direct hooks with `-s/--policy-socket` option, and as an example\nof handling such socket interactions.\n\n[nbrpc.service] and other \\*.service files can be used to setup the script(s)\nto run with systemd, though make sure to tweak Exec-lines and any other paths\nin there first.\n\n`-P/--print-state` can be used to check on all host and address states anytime.\n\nOnce that works, additional instance of the script can be added to run in\nmostly same way, but with following two diffs:\n\n- `-F/--failing-checks` option added, and maybe interval tweaks.\n- Firewall/routing setup to send all traffic of that second instance through\n  whatever workaround route/tunnel that is supposed to be used.\n\nSee info on that option for more details, but gist is that running such instance\ncan help to detect prolonged global service outages and avoid marking hosts as\nblocked if they just don't work anywhere due to that.\n\"host-na-state\" grace-interval should prevent changing state on brief outages without this.\n\nAlso see below for an extended OS routing integration example.\n\n[nbrpc.py]: nbrpc.py\n[nbrpc-policy-cmd.py]: nbrpc-policy-cmd.py\n[nbrpc-policy-nft.py]: nbrpc-policy-nft.py\n[nbrpc.service]: nbrpc.service\n\n\n\u003ca name=hdr-check_list_file_format\u003e\u003c/a\u003e\n## Check list file format\n\nShould be a space/newline-separated list of hostnames to check.\n\nEach spec can be more than just hostname: `hostname[\u003epolicy][:check][=expected-result]`\n\n- `hostname` - hostname or address to use with getaddrinfo() for each check.\n\n    It almost always makes sense to only use names for http(s) checks, as sites\n    tend to change IPs, and names are required for https, SNI and proper vhost\n    responses anyway.\n\n- `check` - type of check to run.\n\n    Currently supported checks: `https`, `http`, `dns`. Default: `https`.\n\n    http/https checks can also have a pre-encoded URL path included, e.g.\n    `https/url/path...`, to query that for more useful response status code.\n    If there's `=` in URL path, replace/escape it with `==`.\n\n    \"dns\" check is a no-op to track IPs for zone-files output or other purposes.\n\n- `expected-result` - for http(s) checks - response code(s) to treat as an OK result,\n  with anything else considered a failure, separated by slash (\"/\"). Default is 200/301/302.\n\n    Special `na` value will always return failure for any check without running it.\n\n- `policy` - how to combine conflicting check results for different host addresses.\n\n    This value should look like `reroute-policy.dns-flags`, where both\n    dot-separated parts are optional.\n\n    `reroute-policy` can be one of the following values:\n\n    - `af-any` - host considered ok if all addrs on either IPv4 or IPv6 address family (AF) are okay.\n    - `af-all` - any blocked addr on any AF = host considered N/A.\n    - `af-pick` - reroute all addrs of AF(s) that have any of them blocked.\n    - `pick` - reroute individual addrs that appear to be blocked, instead of per-host/AF policy.\n    - `noroute` - always return same \"ok\" for routing policy purposes.\n\n    `dns-flags` part is a combination of any number of one-char DNS-filtering\n    flags from the following list:\n\n    - `4` - only resolve and use/check IPv4 A records/addrs for host.\n    - `6` - only resolve/use/check IPv6 AAAA addresses.\n    - `D` - print only records for directly-accessible addrs of this host.\n    - `N` - only print records for inaccessible/rerouted addrs.\n    - `L` - print only latest records IPs from last getaddrinfo() for host, not any earlier ones.\n    - `1` - only take addrs from last getaddrinfo() into account for updating host state.\n    - `R` - always print records in a random (shuffled) order.\n\n    Where \"print\" flags are only relevant when using `-Z/--zone-for` option.\n\n    Any combination of these should work - for example `pick.6`, `LD4`,\n    `af-all`, `af-pick.NL` - but using some DNS flags like `46` together\n    makes them negate each other.\n\n    Default value is `af-all`.\n    Can be changed via `-p/--check-list-default-policy` script option.\n\nEmpty lines are fine, anything after # to the end of the line is ignored as comment.\n\nSimple Example:\n\n```\n## Twitter and some of its relevant subdomains\ntwitter.com\nabs.twimg.com=400 api.twitter.com=404 # some endpoints don't return 200\n\n## Random other check-type examples\noldsite.com:http\nfickle-site.net=200/503\nhttpbin.org:https/status/478=478\n\n## Policy examples\nwww.wikipedia.org\u003epick.RL\nabcdefg.cloudfront.net\u003eLD:https/api=400\n\n## Always route-around Lets-Encrypt OCSP requests for more privacy/reliability\n# https://letsencrypt.org/docs/lencr.org/\nocsp.int-x3.letsencrypt.org=na r3.o.lencr.org=na\n```\n\nThese config files can be missing, created, removed or changed on the fly,\nwith their mtimes probed on every check interval, and contents reloaded as needed.\n\nAt least one `-f/--check-list-file` option is required, even with nx path.\n\n\n\u003ca name=hdr-setup_example_with_linux_policy_routing\u003e\u003c/a\u003e\n## Setup example with linux policy routing\n\nRelatively simple way to get this tool to control network is to have it run\non some linux router box and tweak its routing logic directly for affected IPs,\nrouting traffic to those through whatever tunnel, for example.\n\nThis is generally called \"Policy Routing\", and can be implemented in a couple\ndifferent ways, more obvious of which are:\n\n- Add custom routes to each address that should be indirectly accessible\n  to the main routing table.\n\n    E.g. `ip ro add 216.58.211.14 via 10.10.0.1 dev mytun`, with 10.10.0.1 being\n    a custom tunnel gateway IP on the other end.\n\n    Dead-simple, but can be somewhat messy to manage.\n\n    [ip route] can group/match routes by e.g. \"realm\" tag, so that they can be\n    nuked and replaced all together to sync with desired state.\n\n    It also has `--json` option, which can help managing these from scripts,\n    but it's still a suboptimal mess for this purpose.\n\n- Add default tunnel gateway to a separate routing table,\n  and match/send connections to that using linux [ip rules] table:\n\n    ```\n    ip ro add default via 10.10.0.1 dev mytun table vpn\n    ip ru add to 216.58.211.14 lookup vpn\n    ```\n\n    (table \"vpn\" can be either defined in `/etc/iproute2/rt_tables` or referred\n    to by numeric id instead)\n\n    Unlike with using default routing table above, this gives more flexibility wrt\n    controlling how indirect traffic is routed - separate table can be tweaked\n    anytime, without needing to flush and replace every rule for each IP-addr.\n\n    It's still sequential rule-matching, lots of noise (moved from ip-route to\n    ip-rule table), and messy partial updates.\n\n- Match and mark packets using powerful firewall capabilities (old iptables,\n  nftables or ebtables) and route them through diff tables based on that:\n\n    ```\n    ip ro add default via 10.10.0.1 dev mytun table vpn\n    ip ru add fwmark 0x123 lookup vpn\n    nft add rule inet filter pre iifname mylan ip daddr 216.58.211.14 mark set 0x123\n    ```\n\n    It's another layer of indirection, but [nftables] (linux firewall) has proper\n    IP sets with atomic updates and replacement to those.\n\n    So that one marking rule can use nftables set - e.g.\n    `nft add rule inet filter pre iifname mylan ip daddr @nbrpc mark set 0x123` -\n    and those three rules are basically all you ever need for dynamic policy routing.\n\n    Just gotta add/remove IPs in @nbrpc to change routing decisions, all being\n    neatly contained in that set, with very efficient packet matching,\n    and infinitely flexible too if necessary (i.e. not only by dst-ip, but pretty\n    much anything, up to and including running custom BPF code on packets).\n\n    Having decisions made at the firewall level also allows to avoid this routing\n    to affect the script itself - \"prerouting\" hook will already ensure that, as\n    it doesn't affect locally-initiated traffic, but with e.g. \"route\" hook that\n    does, something trivial like `skuid nbrpc` can match and skip it by\n    user/group or cgroup where it's running under systemd.\n\n[nbrpc-policy-nft.py] script in this repo can be used with that last approach,\ncan run separately from the main checker script (with [cap_net_admin] to tweak\nfirewall), replacing specified IPv4/IPv6 address sets on any changes.\n\nGeneral steps for this kind of setup:\n\n- Some kind of external tunnel, for example:\n\n    ```\n    ip link add mytun type gre local 12.34.56.78 remote 98.76.54.32\n    ip addr add 10.10.0.2/24 dev mytun\n    ip addr add fddd::10:2/120 dev mytun\n    ip link set mytun up\n    ```\n\n    Such GRE tunnel is nice for wrapping any IPv4/IPv6/eth traffic to go between\n    two existing IPs, but not secure to go over internet by any means - something\n    like [WireGuard] is much better for that (and GRE can go over some pre-existing\n    wg link too!).\n\n- Policy routing setup, where something can be flipped for IPs to switch between\n  direct/indirect routes:\n\n    ```\n    nft add chain inet filter route '{ type route hook output priority mangle; }'\n    nft add chain inet filter pre '{ type filter hook prerouting priority raw; }'\n    nft add chain inet filter vpn-mark;\n\n    nft add set inet filter nbrpc4 '{ type ipv4_addr; }'\n    nft add set inet filter nbrpc6 '{ type ipv6_addr; }'\n\n    nft add rule inet filter route oifname mywan jump vpn-mark  ## own traffic\n    nft add rule inet filter pre iifname mylan jump vpn-mark    ## routed traffic\n\n    ## Exception for nbrpc script itself\n    nft add rule inet filter vpn-mark skuid nbrpc ct mark set 0x123 return\n    nft add rule inet filter vpn-mark ct mark == 0x123 return   ## icmp/ack/rst after exit\n\n    nft add rule inet filter vpn-mark ip daddr @nbrpc4 mark set 0x123\n    nft add rule inet filter vpn-mark ip6 daddr @nbrpc6 mark set 0x123\n\n    ip -4 ro add default via 10.10.0.1 dev mytun table vpn\n    ip -4 ru add fwmark 0x123 lookup vpn\n    ip -6 ro add default via fddd::10:1 dev mytun table vpn\n    ip -6 ru add fwmark 0x123 lookup vpn\n    ```\n\n    \"nbrpc4\" and \"nbrpc6\" nftables sets in this example will have a list of IPs\n    that should be routed through \"vpn\" table and GRE tunnel gateway there,\n    add snat/masquerade rules after that as needed.\n\n    \"type route\" hook will also mark/route host's own traffic for matched IPs\n    (outgoing connections from its OS/pids), not just stuff forwarded through it.\n\n    Firewall rules should probably be in nftables.conf file, and have a hook\n    sending SIGHUP to nbrpc on reload, to have it re-populate sets there as well,\n    while \"ip\" routes/rules configured in whatever network manager, if any.\n\n    Reverse \"skuid\" match should be applied to script instance running with\n    `-F/--failing-checks`, if it is used, to have all its traffic routed through\n    \"vpn\" table, as opposed to the main instance.\n\n- Something to handle service availability updates from main script\n  and update routing policy:\n\n    ```\n    cd ~nbrpc\n    capsh --caps='cap_net_admin+eip cap_setpcap,cap_setuid,cap_setgid+ep' \\\n      --keep=1 --user=nbrpc --addamb=cap_net_admin --shell=/usr/bin/python -- \\\n      ./nbrpc-policy-nft.py -s nft.sock -4 :nbrpc4 -6 :nbrpc6 -p\n    ```\n\n    Long capsh command (shipped with libcap) runs nbrpc-policy-nft.py with\n    [cap_net_admin] to allow it access to the firewall without full root.\n    Same as e.g. `AmbientCapabilities=CAP_NET_ADMIN` with systemd.\n\n- Main nbrpc.py service running checks with its own db:\n\n    ```\n    cd ~nbrpc\n    su-exec nbrpc ./nbrpc.py --debug -f hosts.txt -Ssx nft.sock\n    ```\n\n    Can safely run with some unprivileged uid and/or systemd/lsm sandbox setup,\n    only needing to access nft.sock unix socket of something more privileged,\n    without starting any fancy sudo/suid things directly.\n\n- Setup tunnel endpoint and forwarding/masquerading on the other side, if missing.\n\nThat is to use checked services' status to tweak OS-level routing though,\nand failover doesn't have to be done this way - some exception-list can be used\nin a browser plugin to direct it to use proxy server(s) for specific IPs,\nor something like [Squid] can be configured as a transparent proxy with its own\nconfig of rules, or maybe this routing info can be relayed to a dedicated router\nappliance.\n\nMain nbrpc script doesn't care either way - give it a command or socket to feed\nstate/updates into and it should work.\n\n[curl]: https://curl.se/\n[ip route]: https://man.archlinux.org/man/ip-route.8.en\n[ip rules]: https://man.archlinux.org/man/ip-rule.8.en\n[nftables]: https://nftables.org/\n[WireGuard]: https://www.wireguard.com/\n[cap_net_admin]: https://man.archlinux.org/man/capabilities.7.en\n[Squid]: http://www.squid-cache.org/\n\n\n\u003ca name=hdr-related_links__tips__info_and_trivia\u003e\u003c/a\u003e\n## Related links, tips, info and trivia\n\n- Main script keeps all its state in an sqlite db file (using WAL mode),\n  isolating all state changes in exclusive db transactions, so should be fine to\n  run multiple instances of it with the same source files and db anytime.\n\n    Potential quirks when doing that can be:\n\n    - Changing check types for host(s) while these checks are running might cause\n      address and host state to be set based on type/result info from when that\n      check was started, which should be fixed by the next run.\n\n    - If this script is used with giant lists/DBs or on a slow host/storage\n      (like an old RPi1 with slow SD card under I/O load), db transactions can\n      take more than hardcoded sqlite locking timeout (60 seconds), and abort\n      with error after that.\n\n    There should be no reason to run concurrent instances of the script normally,\n    with only exception being various manual checks and debug-runs,\n    using e.g. `-P/--print-state`, `-u/--update-host` and such options.\n\n- Even though examples here have \"nft add rule\" commands for simplicity,\n  it's generally a really bad idea to configure firewall like that - use same\n  exact \"add rule\" commands or rule-lines in table blocks within a single\n  nftables.conf file instead.\n\n    Difference is that conf file is processed and applied/rejected atomically,\n    so that firewall can't end up in an arbitrary broken state due to some rules\n    failing to apply - either everything gets configured as specified, or error\n    is signaled and nothing is changed.\n\n- Masquerading traffic going through the tunnel can be done in the usual way,\n  via forward+reverse traffic-matching rules in the \"forward\" hook and\n  \"masquerade\" or \"snat\" rule applied by the \"nat\" hook.\n\n    In the setup example above, given that relevant outgoing traffic should\n    already be marked for routing, it can be matched by that mark, or combined\n    with iface names anyway:\n\n    ```\n    nft add rule inet filter forward iifname lan oifname mytun cm mark 0x123 accept\n    nft add rule inet filter forward iifname mytun oifname lan accept\n    nft add rule inet nat postrouting oifname mytun cm mark 0x123 masquerade\n    ```\n\n- Tunnels tend to have lower MTU than whatever endpoints might have set on their\n  interfaces, so [clamping TCP MSS via nftables] is usually a good idea:\n\n    ```\n    nft add rule inet filter forward tcp flags syn tcp option maxseg size set rt mtu\n    ```\n\n    This can be tested via e.g. `ping -4M do -s $((1500-28)) somehost.net`\n    (1500B MTU - 8B ICMP header - 20B IPv4 header) plus the usual tcpdump to see\n    MSS on TCP connections and actual packet sizes, and it's quite often not what\n    you expect, so always worth checking at least everywhere where tunneling or\n    whatever overlay protocols are involved.\n\n- systemd-networkd will clobber routes and rules defined via iproute2 \"ip\" tools\n  from console or some script by default, at somewhat random times.\n\n    `ManageForeignRoutingPolicyRules=no` and `ManageForeignRoutes=no` options\n    in networkd.conf can be used to disable that behavior, or routes/rules defined\n    via its configuration files properly.\n\n- If some service is hopping between IPs too much, so that nbrpc can't catch-up\n  with it, and occasionally-failing connections are annoying, script has\n  `-Z/--zone-for` option to export local-zone with only A/AAAA records\n  known to it (or some subset - see option description) for regexp-filtered list\n  of known/managed hostnames (can be just `-Z.` to dump all of them).\n\n    Output produced there by default can be used with [Unbound]'s (DNS\n    resolver/cache daemon) `include:` directive, or with [CoreDNS] \"hosts\"\n    directive (picking \\/etc\\/hosts file format with `-z hosts` option),\n    or parsed/converted for other local resolvers from those.\n    Should probably be scheduled via systemd timer\n    (with e.g. `StandardOutput=truncate:...` line) or crontab.\n\n    Note that same DNS resolver with zone overrides shouldn't be used for main\n    nbrpc script itself, which can be easy to fix by e.g. bind-mounting different\n    resolv.conf file (pointing to unrestricted resolver) into its systemd service/container.\n\n- While intended to work around various network disruptions, this stuff can also\n  be used in the exact opposite way - to detect when specific endpoints are\n  accessible and block them - simply by reading \"ok\" result in policy-updates as\n  undesirable (instead of \"na\", adding blocking rules), e.g. in a [pihole]-like scenario.\n\n- [test.sh](test.sh) script can be used to easily check or create any oddball\n  blocking-over-time scenarios and see how logic of the tool reacts to those,\n  coupled with specific configuration or any local code tweaks, and is full of examples.\n\n- [\"Dynamic policy routing to work around internet restrictions\" blog post]\n  with a bit more context and info around this script.\n\n\n[clamping TCP MSS via nftables]:\n  https://wiki.nftables.org/wiki-nftables/index.php/Mangling_packet_headers\n[Unbound]: https://unbound.docs.nlnetlabs.nl/\n[CoreDNS]: https://coredns.io/\n[pihole]: https://pi-hole.net/\n[\"Dynamic policy routing to work around internet restrictions\" blog post]:\n  https://blog.fraggod.net/2022/04/05/dynamic-policy-routing-to-work-around-internet-restrictions.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk-fg%2Fname-based-routing-policy-controller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmk-fg%2Fname-based-routing-policy-controller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmk-fg%2Fname-based-routing-policy-controller/lists"}