{"id":17592932,"url":"https://github.com/travisghansen/metallb-node-route-agent","last_synced_at":"2025-04-30T03:44:16.968Z","repository":{"id":142609496,"uuid":"475053937","full_name":"travisghansen/metallb-node-route-agent","owner":"travisghansen","description":"ensure return path through incoming bgp routers","archived":false,"fork":false,"pushed_at":"2025-04-09T22:18:22.000Z","size":282,"stargazers_count":10,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-30T03:44:11.735Z","etag":null,"topics":["frr","k8s","kubernetes","linux","metallb","routing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/travisghansen.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-03-28T15:03:46.000Z","updated_at":"2025-04-09T22:18:14.000Z","dependencies_parsed_at":null,"dependency_job_id":"899b7fdb-f5eb-46a6-8465-f3d2dec8faf7","html_url":"https://github.com/travisghansen/metallb-node-route-agent","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisghansen%2Fmetallb-node-route-agent","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisghansen%2Fmetallb-node-route-agent/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisghansen%2Fmetallb-node-route-agent/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/travisghansen%2Fmetallb-node-route-agent/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/travisghansen","download_url":"https://codeload.github.com/travisghansen/metallb-node-route-agent/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251638750,"owners_count":21619661,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["frr","k8s","kubernetes","linux","metallb","routing"],"created_at":"2024-10-22T05:44:18.029Z","updated_at":"2025-04-30T03:44:16.943Z","avatar_url":"https://github.com/travisghansen.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Image](https://img.shields.io/docker/pulls/travisghansen/metallb-node-route-agent.svg)\n![Image](https://img.shields.io/github/actions/workflow/status/travisghansen/metallb-node-route-agent/main.yml?branch=master\u0026style=flat-square)\n\n# metallb-node-route-agent\n\nThe purpose of this project is to create routes on the nodes to ensure outbound\n`LoadBalancer` traffic uses the `metallb` bgp peers on the return path (ie:\navoid possible asymmetric routing scenarios).\n\nTo achieve this goal the agent creates/manages a specific routing table and\ndirects all traffic from relevant `LoadBalancer` IPs to route through the\nconfigured `metallb` `peers`.\n\nIt is assumed this application will entirely manage the `TABLE_NAME` routing\ntable along with all rules referencing the table. In other words, any manually\ncreated `rules` or `routes` in the table will be considered 'imposters' and\nremoved.\n\n# configuration\n\nThe container should run in the `host` networking namespace. Additionally you\nmust mount `/etc/iproute2` from the host into the same location in the\ncontainer.\n\nAdditionally if you have multiple paths (bgp peers) you likely want to ensure\nyour nodes have the sysctl `net.ipv4.fib_multipath_use_neigh` set to `1`.\n\n## env vars\n\n- `MAX_RECONCILE_WAIT` - a timer is setup on this interval (ms) to reconcile\n  - default: 60s\n  - minimum value is 1s\n  - `0` disables the feature\n- `TABLE_WEIGHT` - weight of the routing table to manage\n  - default: `20`\n- `PEER_WEIGHT` - weight of each peer added to the routing table\n  - default: 100\n  - all peers have the same weight currently so not super helpful to modify\n- `RULE_PRIORITY` - the priority to give to the managed rules\n  - default: 20\n- `RULE_FWMARK` - the fwmark to give to the managed rules (show be provided in\n  hex format exactly as the `ip` output shows)\n  - default: unset\n- `DESTINATION` - the `dst` network of the route\n  - default: `default`\n- `PRE_RECONCILE_SCRIPT_PATH` - path to script (must be marked executable) to\n  run _before_ the reonciliation happens\n- `POST_RECONCILE_SCRIPT_PATH` - path to script (must be marked executable) to\n  run _after_ the reonciliation happens\n- `METALLB_NAMESPACE` - namespace where `metallb` is running\n  - default: `\"\"`\n  - will fallback to the value in\n    `/var/run/secrets/kubernetes.io/serviceaccount/namespace` if possible\n  - will use `metallb-system` as a last resort\n- `METALLB_CONFIGMAP_NAME` - name of the `metallb` `configmap`\n  - default: `config`\n- `METALLB_STATIC_FILE` - a static file on the filesystem to monitor (mostly\n  for development purposes). If set the k8s watch is disabled entirely.\n- `METALLB_USE_CRDS` - prefer `CRDs` over configmap\n- `LOG_LEVEL` - `error|warn|info|verbose|debug|silly`\n  - default: `info`\n- `CLEANANDEXIT` - if equals `1` then all rules/tables will be deleted and the\n  process will exit\n- `ONESHOT` - if equals `1` then then reconciliation will complete once and the\n  process will exit (useful as a cronjob for example)\n\n# CNI\n\n## cilium\n\nThis project 'just works' with cilium if using hte kube-proxy replacement feature.\n\n## calico\n\nIf using with calico you must run `kube-proxy` in `ipvs` mode. In addition it\nwill likely require very special firewall rules to ensure proper traffic flows.\n\nWithout the rules etc below undesirable traffic flows will occur and\nfunctionality will likely break. Namely Pod (both CNI and HostNetwork) traffic\nmay end up routing to BGP Peers instead of staying local to the cluster.\n\n```\n# 0x14 = 20 in decimal, you may use whatever value you wish however\n# mark *connections* coming from the 'outside' world\n# \n# Should be *before* KUBE-SERVICES\n# 1      625 37918 cali-PREROUTING  all  --  any    any     anywhere             anywhere             /* cali:6gwbT8clXdHdC1b1 */\n# 2        1    60 CONNMARK   all  --  any    any     anywhere             anywhere             match-set KUBE-LOAD-BALANCER dst,dst ! match-set cali40masq-ipam-pools src CONNMARK set 0x14\n# 3     537K   47M KUBE-SERVICES  all  --  any    any     anywhere             anywhere             /* kubernetes service portals */\n#\niptables -t nat -I PREROUTING \\\n  -m set   --match-set KUBE-LOAD-BALANCER dst,dst \\\n  -m set ! --match-set cali40masq-ipam-pools src \\\n  -j CONNMARK --set-mark 0x14\n\n# copy *connection* mark to *packet* mark *before* POSTROUTING/SNAT takes place\niptables -t mangle -I FORWARD -m connmark --mark 0x14 -j CONNMARK --restore-mark\n\n# ensure proper env vars for metallb-nra\nRULE_FWMARK=0x14\n```\n\n# development\n\n```\nsudo -E METALLB_STATIC_FILE=./examples/metallb-config.yaml MAX_RECONCILE_WAIT=5000 node agent.js\n\ndocker build --pull -t foobar .\ndocker run --rm -ti --net=host -v /etc/iproute2:/etc/iproute2 foobar bash\n\n# create table\n# test table exists\n# exit 0   = exists\n# exit 2   = exists, no entries\n# exit 255 = not exists\nip route show table metallb-nra\necho 20 metallb-nra \u003e\u003e /etc/iproute2/rt_tables\n\n\n# upsert routes to the table\n# network parsing/calculation needed in this step\nip route replace default via 172.28.4.130 table metallb-nra\n\n# note that linux does a hash-based/tuple algorithm\n# https://serverfault.com/questions/696675/multipath-routing-in-post-3-6-kernels\n# https://docs.kernel.org/networking/nexthop-group-resilient.html\nip route replace default table metallb-nra \\\n    nexthop via 172.28.4.130 weight 1 \\\n    nexthop via 172.28.4.131 weight 1\n\n# add rule(s)\n# test if the rule is already present? do not create duplicates\n# ip rule show\nip rule add from 172.28.42.0/24 lookup metallb-nra\n...\n\nip rule add from \u003cservice network\u003e lookup metallb-nra\nip route add default via \u003cfrr ip\u003e table metallb-nra\n\n\n# review entries\nip -d rule show table metallb-nra\nip -d route show table metallb-nra\n\n# remove rules\nwhile ip rule delete from 0/0 to 0/0 table metallb-nra 2\u003e/dev/null; do true; done\n\n\nip route flush cache\n```\n\n# TODO\n\n- introduce a one-shot execution style to set/wipe state\n- use `ip -batch` for operations?\n- use `nexthop groups`?\n- use a proper `netlink` library to manage the rules\n  - https://github.com/vishvananda/netlink\n  - https://github.com/hariguchi/iproute\n  - https://www.npmjs.com/package/netlink\n  - https://www.npmjs.com/package/node-netlink\n  - https://github.com/k13-engineering/node-rtnetlink\n\n# links\n\n- https://serverfault.com/questions/696675/multipath-routing-in-post-3-6-kernels\n- https://docs.kernel.org/networking/nexthop-group-resilient.html\n- http://manpages.ubuntu.com/manpages/trusty/man8/ip-route.8.html\n- https://manpages.ubuntu.com/manpages/trusty/man8/ip-rule.8.html\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisghansen%2Fmetallb-node-route-agent","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftravisghansen%2Fmetallb-node-route-agent","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftravisghansen%2Fmetallb-node-route-agent/lists"}