{"id":20446953,"url":"https://github.com/r-caamano/ebpf-tproxy-splicer","last_synced_at":"2025-05-08T19:36:34.117Z","repository":{"id":61253126,"uuid":"546691249","full_name":"r-caamano/ebpf-tproxy-splicer","owner":"r-caamano","description":"This is a project to develop an ebpf program that uses ebpf tc to redirect ingress ipv4 udp/tcp flows toward specific dynamically created sockets and acts as a stateful firewall.  ","archived":false,"fork":false,"pushed_at":"2024-04-02T12:26:47.000Z","size":807,"stargazers_count":30,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-04-02T13:42:56.418Z","etag":null,"topics":["ebpf","firewall","linux","openziti","proxy","tc","tproxy"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/r-caamano.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-10-06T13:43:05.000Z","updated_at":"2024-04-02T13:43:02.081Z","dependencies_parsed_at":"2024-04-02T13:42:57.910Z","dependency_job_id":"5854488a-7034-48c8-ab10-e6170742fb0c","html_url":"https://github.com/r-caamano/ebpf-tproxy-splicer","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-caamano%2Febpf-tproxy-splicer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-caamano%2Febpf-tproxy-splicer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-caamano%2Febpf-tproxy-splicer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/r-caamano%2Febpf-tproxy-splicer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/r-caamano","download_url":"https://codeload.github.com/r-caamano/ebpf-tproxy-splicer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224764248,"owners_count":17365885,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ebpf","firewall","linux","openziti","proxy","tc","tproxy"],"created_at":"2024-11-15T10:23:51.814Z","updated_at":"2024-11-15T10:23:52.526Z","avatar_url":"https://github.com/r-caamano.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ARCHIVED\n  ## Project moved to https://github.com/netfoundry/zfw\n\n                                   \n\n## Introduction\n--- \nThis is a project to develop an eBPF program that utilizes tc-bpf to act as a statefull ingress FW and to redirect \ningress ipv4 udp/tcp flows toward dynamically created sockets that correspond to zero trust based services on OpenZiti\nedge-routers. Those interested on how to setup an openziti development environment should visit \nhttps://github.com/openziti/ziti. \nAlso note this is eBPF tc based so interception only occurs for traffic ingressing on the interface that the eBPF program\nis attached to. To intercept packets generated locally by the router itself the eBPF program would need to be attached to\nthe loopback interface. The eBPF program also provides stateful inbound firewalling and only allows ssh, dhcp and arp\nbypass by default. Initially the program will allow ssh to any address inbound however after the first tproxy mapping is\ninserted by the map_update tool it will only allow ssh addressed to the IP address of the interface that tc has loaded the\neBPF program.  All other traffic must be configured as a service in an OpenZiti Controller which then informs the edge-router\nwhich traffic flows to accept. The open ziti edge-router then uses the map_update user space app to insert rules to\nallow traffic in on the interface tc is running on. For those interested in additional background on the project please visit: \nhttps://openziti.io/using-ebpf-tc-to-securely-mangle-packets-in-the-kernel-and-pass-them-to-my-secure-networking-application.  \n\n\n## Build\n---\n[To build from source. Click here!](./BUILD.md)\n\n## Management After Deployment\n---\n\n### Attaching to interface\n\n```bash   \nsudo map_update --set-tc-filter \u003cinterface name\u003e --object-file=tproxy_splicer.o --direction=ingress\nsudo map_update --set-tc-filter \u003cinterface name\u003e--object-file=outbound_track.o --direction=egress //optional only if firewalling subtending devices see below\nsudo ufw allow in on \u003cinterface name\u003e to any\n```\n\nebpf will now take over firewalling this interface and only allow ssh, dhcp and arp till ziti\nservices are provisioned as inbound intercepts via the map_udate app. Router will statefully allow responses to router\ninitiated sockets as well. tc commands above do not survive reboot so would need to be added to startup service / script.\n\nA new addtion is firewall support for subtending devices for two interface scenarios i.e.\nexternal and trusted.\n\n    external inet \u003c----\u003e (ens33)[ebpf-router](ens37) \u003c----\u003e trusted clients\n\n    with tproxy-splicer.o applied ingress on ens33 and oubound_track.o applied egress on ens33 the router will\n    statefully track outbound udp and tcp connections on ens33 and allow the associated inbound traffic.  While\n    running in this mode it does not make sense to add ziti tproxy rules and is meant for running as a traditional fw.\n    As be for you can also create passthrough FW rules (set -t --tproxy-port to 0) which would also make sense in the mode for\n    specific internet initiated traffic you might want to allow in.\n\n    TCP:\n        If the tcp connections close gracefully then the entries will remove upon connection closure. \n        if not then there is a 60 minute timeout that will remove the in active state if no traffic seen\n        in either direction.\n\n    UDP:\n        State will remain active as long as packets tuples matching SRCIP/SPORT/DSTIP/DPORT are seen in\n        either direction within 30 seconds.  If no packets seen in either dorection the state will expire.\n        If an external packet enters the interface after expire the entry will be deleted.  if an egress\n        packet fined a matching expired state it will return the state to active.\n\n    In order to support this per interface rule awareness was added which allows each port range within a prefix\n    to match a list of connected interfaces.  On a per interface basis you can decide to honor that list or not via\n    a per-prefix-rules setting in the following manner via the map_update utility\n    \n    singly:\n    ```\n    sudo map_update -P \u003cifname\u003e\n    ```\n    or \n\n    all interfaces:\n    ```\n    sudo map_update -P all\n    ```\n\n    In order to assign 1 to 3 interfaces to a rule you would use the new -N option in combination with the -I i.e.\n    to associate the rule to end37 and lo:\n\n    ```\n    sudo map_update -I -c 172.16.31.0 -m 24 -l 443 -h 443 -t 44000 -p tcp -N ens37 -N lo\n    ```\n\n### Openziti Ingress\n\nTesting with ziti-router after attaching. Build a ziti network first and create services as explained at [Host It Anywhere](https://docs.openziti.io/docs/learn/quickstarts/network/hosted/)\n\nGrab the edge router binary `(\u003e= v0.27.3)` at [open ziti](https://github.com/openziti/ziti/releases).\n\n```bash\n# Copy the user space map program to folder in $PATH i.e\nsudo cp map_update /usr/bin\n\n# In the router config.yml set tunnel mode to ebpf i.e.   \n- binding: tunnel\n  options:\n    mode: tproxy:/path/to/map_update\n\n# Run edge router command \n# Note: assumption - ziti-router and config.yml in PATH\nsudo ziti-router run config.yml\n```\n\n### Detaching from interface:\n\n```bash\nsudo map_update --set-tc-filter \u003cinterface name\u003e  --direction \u003cingress | egress\u003e --disable\n```\n\n## Ebpf Map User Space Management\n---\nExample: Insert map entry to direct SIP traffic destined for 172.16.240.0/24\n\n```bash\nUsage: ./map_update -I \u003cip dest address or prefix\u003e -m \u003cprefix length\u003e -l \u003clow_port\u003e -h \u003chigh_port\u003e -t \u003ctproxy_port\u003e -p \u003cprotocol\u003e\n\nsudo ./map_update -I -c 172.16.240.0 -m 24 -l 5060 -h 5060 -t 58997 -p udp\n```\n\nAs mentioned earlier if you add -r, --route as argument the program will add 172.16.240.0/24 to the \"lo\" interface if it\ndoes not overlap with an external LAN interface subnet.\n\nExample: Disable ssh from interface.\n\nThis will disable default ssh action to pass to ip of local interface and then fall through to rule check instead where a more specific rule could\nbe applied.  This is a per interface setting and can be set for all interfaces except loopback.\n\n```bash\nUsage: ./map_update -x \u003cinterface-name\u003e | all\n\nsudo sudo ./map_update -x ens33\n```\n\n\nExample: Insert map entry to with source filteing to only allow rule for ip source 10.1.1.1/32.\n\n```bash\nUsage: ./map_update -I -c \u003cip dest address or prefix\u003e -m \u003cdest prefix len\u003e -o \u003corigin address or prefix\u003e -n \u003corigin prefix len\u003e -l \u003clow_port\u003e -h \u003chigh_port\u003e -t \u003ctproxy_port\u003e -p \u003cprotocol\u003e\n\nsudo sudo ./map_update -I -c 172.16.240.0 -m 24 -o 10.1.1.1 -n 32  -p tcp -l 22 -h 22 -t 0\n```\n\nExample: Insert FW rule for local router tcp listen port 443 where local router's tc interface ip address is 10.1.1.1\nwith tproxy_port set to 0 signifying local connect rule\n\n```bash\nsudo ./map_update -I -c 10.1.1.1 -m 32 -l 443 -h 443 -t 0 -p tcp  \n```\n\nExample: Monitor ebpf trace messages\n\n```\nsudo map_update -v all\nsudo cat /sys/kernel/debug/tracing/trace_pipe\n  \n\u003cidle\u003e-0       [007] dNs.. 167940.070727: bpf_trace_printk: ens33\n\u003cidle\u003e-0       [007] dNs.. 167940.070728: bpf_trace_printk: source_ip = 0xA010101\n\u003cidle\u003e-0       [007] dNs.. 167940.070728: bpf_trace_printk: dest_ip = 0xAC10F001\n\u003cidle\u003e-0       [007] dNs.. 167940.070729: bpf_trace_printk: protocol_id = 17\n\u003cidle\u003e-0       [007] dNs.. 167940.070729: bpf_trace_printk: tproxy_mapping-\u003e5060 to 59423\n\n\u003cidle\u003e-0       [007] dNs.. 167954.255414: bpf_trace_printk: ens33\n\u003cidle\u003e-0       [007] dNs.. 167954.255414: bpf_trace_printk: source_ip = 0xA010101\n\u003cidle\u003e-0       [007] dNs.. 167954.255415: bpf_trace_printk: dest_ip = 0xAC10F001\n\u003cidle\u003e-0       [007] dNs.. 167954.255415: bpf_trace_printk: protocol_id = 6\n\u003cidle\u003e-0       [007] dNs.. 167954.255416: bpf_trace_printk: tproxy_mapping-\u003e22 to 39839\n\n```\nExample: Remove previous entry from map\n```bash\nUsage: ./map_update -D -c \u003cip dest address or prefix\u003e -m \u003cprefix len\u003e -l \u003clow_port\u003e -p \u003cprotocol\u003e\n\nsudo ./map_update -D -c 172.16.240.0 -m 24 -l 5060 -p udp\n```\n\nExample: Remove all entries from map\n```\nUsage: ./map_update -F\n\nsudo ./map_update -F\n```\n\nExample: List all rules in map\n```\nUsage: ./map_update -L\n\nsudo ./map_update -L\n\ntarget     proto    origin              destination               mapping:                                                   interface list\n------     -----    ---------------     ------------------        --------------------------------------------------------- ----------------\nTPROXY     tcp      0.0.0.0/0           10.0.0.16/28              dpts=22:22                TPROXY redirect 127.0.0.1:33381  [ens33,lo]\nTPROXY     tcp      0.0.0.0/0           10.0.0.16/28              dpts=30000:40000          TPROXY redirect 127.0.0.1:33381  []\nTPROXY     udp      0.0.0.0/0           172.20.1.0/24             dpts=5000:10000           TPROXY redirect 127.0.0.1:59394  []\nTPROXY     tcp      0.0.0.0/0           172.16.1.0/24             dpts=22:22                TPROXY redirect 127.0.0.1:33381  []\nTPROXY     tcp      0.0.0.0/0           172.16.1.0/24             dpts=30000:40000          TPROXY redirect 127.0.0.1:33381  []\nPASSTHRU   udp      0.0.0.0/0           192.168.3.0/24            dpts=5:7                  PASSTHRU to 192.168.3.0/24       []\nPASSTHRU   udp      10.1.1.1/32         192.168.100.100/32        dpts=50000:60000          PASSTHRU to 192.168.100.100/32   []\nPASSTHRU   tcp      10.230.40.1/32      192.168.100.100/32        dpts=60000:65535          PASSTHRU to 192.168.100.100/32   []\nTPROXY     udp      0.0.0.0/0           192.168.0.3/32            dpts=5000:10000           TPROXY redirect 127.0.0.1:59394  []\nPASSTHRU   tcp      0.0.0.0/0           192.168.100.100/32        dpts=60000:65535          PASSTHRU to 192.168.100.100/32   []\n```\nExample: List rules in map for a given prefix and protocol\n```bash\n# Usage: ./map_update -L -c \u003cip dest address or prefix\u003e -m \u003cprefix len\u003e -p \u003cprotocol\u003e\n  \nsudo map_update -L -c 192.168.100.100 -m 32 -p udp\n  \ntarget     proto    origin           destination              mapping:                                                  interface list\n------     -----    --------         ------------------       --------------------------------------------------------- ------------------    \nPASSTHRU   udp      0.0.0.0/0        192.168.100.100/32       dpts=50000:60000 \t      PASSTHRU to 192.168.100.100/32     []\n``` \n\nExample: List rules in map for a given prefix\n```bash\n# Usage: ./map_update -L -c \u003cip dest address or prefix\u003e -m \u003cprefix len\u003e -p \u003cprotocol\u003e\n\nsudo map_update -L -c 192.168.100.100 -m 32\n\ntarget     proto    origin           destination              mapping:                                                  interface list\n------     -----    --------         ------------------       --------------------------------------------------------- -------------------\nPASSTHRU   udp      0.0.0.0/0        192.168.100.100/32       dpts=50000:60000 \t      PASSTHRU to 192.168.100.100/32     []\nPASSTHRU   tcp      0.0.0.0/0        192.168.100.100/32       dpts=60000:65535\t      PASSTHRU to 192.168.100.100/32     []\n```\nExample: List all interface setting\n```bash\nUsage: ./map_update -L -E\n\nsudo ./map_update -E\n\nlo:\n--------------------------\nicmp echo               :1\nverbose                 :0\nssh disable             :0\nper interface           :0\ntc ingress filter       :1\ntc egress filter        :0\n--------------------------\n\nens33:\n--------------------------\nicmp echo               :0\nverbose                 :1\nssh disable             :1\nper interface           :1\ntc ingress filter       :1\ntc egress filter        :1\n--------------------------\n\nens37:\n--------------------------\nicmp echo               :0\nverbose                 :0\nssh disable             :0\nper interface           :0\ntc ingress filter       :1\ntc egress filter        :0\n--------------------------\n```\n\nExample: Remove all tc-ebpf on router\n```bash\nUsage: ./map_update -Q,--disable-ebpf\n\nsudo map_update --disable-ebpf\ntc parent del : lo\ntc parent del : ens33\ntc parent del : ens37\nremoving /sys/fs/bpf/tc/globals/zt_tproxy_map\nremoving /sys/fs/bpf/tc/globals/diag_map\nremoving /sys/fs/bpf/tc/globals/ifindex_ip_map\nremoving /sys/fs/bpf/tc/globals/tuple_count_map\nremoving /sys/fs/bpf/tc/globals/prog_map\nremoving /sys/fs/bpf/tc/globals/udp_map\nremoving /sys/fs/bpf/tc//globals/matched_map\nremoving /sys/fs/bpf/tc/globals/tcp_map\n```\n\n\n## Additional Distro testing\n---\n\nFedora 36 kernel 6.0.5-200\n\nOn fedora I found that NetworkManager interferes with eBPF socket redirection and can\nbe unpredictable so belowis what I changed to get it working consistently. Other less\nintrusive methods not requiring removal of NM might also be possible.\n\n```bash\nsudo yum install network-scripts\nsudo systemctl enable network\nsudo yum remove NetworkManager\n\nsudo vi /etc/sysconfig/network-scripts/ifcfg-eth0\n\n# if eth0 will be dhcp then something like:\nBOOTPROTO=dhcp\nDEVICE=eth0\nONBOOT=yes\n\n# or if static\nBOOTPROTO=static\nIPADDR=192.168.61.70\nNETMASK=255.255.255.0\nDEVICE=eth1\nONBOOT=yes\n```\n\nThe following grub change is only necessary on systems that do not use ethX naming by\ndefault like vmware. this changes fedora back to using ethX for interface naming network-scripts looks for this nomenclature and will fail DHCP otherwise\n\n```bash \nsudo vi /etc/default/grub\n\n# change:\nGRUB_CMDLINE_LINUX=\"rd.lvm.lv=fedora_fedora/root rhgb quiet\"\n\n# to:\nGRUB_CMDLINE_LINUX=\"rd.lvm.lv=fedora_fedora/root rhgb quiet net.ifnames=0 biosdevname=0\"\n\n# then:\nsudo grub2-mkconfig -o /boot/grub2/grub.cfg\n```\nThis updates dhcp script to dynamically update systemd-resolved on per interface resolver\n\n```bash\nsudo vi /usr/sbin/dhclient-script\n\n# change:\nif [ -n \"${new_domain_name_servers}\" ]; then\n    for nameserver in ${new_domain_name_servers} ; do\n        echo \"nameserver ${nameserver}\" \u003e\u003e \"${rscf}\"\n    done\n\n# to:   \nif [ -n \"${new_domain_name_servers}\" ]; then\n    for nameserver in ${new_domain_name_servers} ; do\n        echo \"nameserver ${nameserver}\" \u003e\u003e \"${rscf}\"\n        systemd-resolve --interface \"${interface}\" --set-dns \"${nameserver}\"\n    done\n````\n\n## Analisys\n---\n\n                                                     DIAGRAMS\n\n![Diagram](packet-flow.drawio.png) \n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr-caamano%2Febpf-tproxy-splicer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fr-caamano%2Febpf-tproxy-splicer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fr-caamano%2Febpf-tproxy-splicer/lists"}