{"id":16161426,"url":"https://github.com/gamemann/xdp-access-last-byte","last_synced_at":"2025-03-18T22:30:42.117Z","repository":{"id":198580270,"uuid":"701080217","full_name":"gamemann/Xdp-Access-Last-Byte","owner":"gamemann","description":"Repository to store information accessing the last byte of a packet in BPF and XDP.","archived":false,"fork":false,"pushed_at":"2024-07-10T09:16:44.000Z","size":15,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-02-28T12:30:01.519Z","etag":null,"topics":["bpf","byte","data","express","last","lastbyte","network","network-programming","networking","packet","path","payload","processing","xdp"],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/gamemann.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-05T21:52:50.000Z","updated_at":"2025-02-16T12:55:25.000Z","dependencies_parsed_at":"2024-10-27T19:16:08.335Z","dependency_job_id":"6b33f2ed-a7ab-4c8d-953d-0902cb5f1b93","html_url":"https://github.com/gamemann/Xdp-Access-Last-Byte","commit_stats":null,"previous_names":["gamemann/xdp-access-last-byte"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gamemann%2FXdp-Access-Last-Byte","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gamemann%2FXdp-Access-Last-Byte/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gamemann%2FXdp-Access-Last-Byte/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/gamemann%2FXdp-Access-Last-Byte/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/gamemann","download_url":"https://codeload.github.com/gamemann/Xdp-Access-Last-Byte/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243950813,"owners_count":20373664,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bpf","byte","data","express","last","lastbyte","network","network-programming","networking","packet","path","payload","processing","xdp"],"created_at":"2024-10-10T02:25:13.876Z","updated_at":"2025-03-18T22:30:41.849Z","avatar_url":"https://github.com/gamemann.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"I'm creating this repository to store my findings on accessing the last byte of data in a packet within [XDP](https://www.iovisor.org/technology/xdp).\n\n## Update (7-10-24)\nI completely forgot about a solution to accessing the last byte of data in XDP I used a long time ago. Please check out my XDP Forwarding project's code [here](https://github.com/gamemann/XDP-Forwarding/blob/master/src/xdp_prog.c#L181) which shows how I did this (in my case, I added four bytes of data to the end of the packet that was set to the client's IP address). You may also check out [this mailing list thread](https://lore.kernel.org/bpf/CANzUK5-g9wLiwUF88em4uVzMja_aR4xj9yzMS_ZObNKjvX6C6g@mail.gmail.com/) which is where I found the solution.\n\n## Building\nYou can use the `make` command to quickly build this project. Otherwise, you can use the following commands.\n\n```bash\nclang -O2 -g -target bpf -o build/last_one.o -c src/last_one.c\nclang -O2 -g -target bpf -o build/last_two.o -c src/last_two.c\n```\n\n## Attaching The XDP Programs\nYou can attempt to attach the XDP object files after building via `make` by performing the following command as root (or using `sudo`).\n\n```bash\nip link set \u003cinterfaceName\u003e xdp obj build/\u003cprog\u003e.o section xdp_prog\n```\n\n* `\u003cinterfaceName\u003e` - The interface name that you want to attach the XDP program to (`ip a`, `ip link`, and `ifconfig` are commands to list interface names in most Linux distros).\n* `\u003cprog\u003e` - The test program to attach after building via `make` (e.g. `last_one`, `last_two`, or `last_three`).\n\n## Findings So Far\n### `last_one.c`\nIn our [last_one.c](./src/last_one.c) XDP program, we try to use `data` and `data_end` to retrieve the offset to the last byte of data in the packet. We then store the byte in our `last` pointer. Afterwards, we check if the last byte is within the packet's bounds by comparing it to `data` and `data_end`.\n\n```C\n#include \u003clinux/bpf.h\u003e\n\n#include \"common.h\"\n\nSEC(\"xdp_prog\")\nint prog(struct xdp_md* ctx) {\n    // Initialize data and data_end.\n    void* data = (void*)(long)ctx-\u003edata;\n    void* data_end = (void*)(long)ctx-\u003edata_end;\n\n    // Retrieve the full length of our packet to use as an offset to the last byte of data from the start of our packet using data_end and data.\n    __u16 off = (__u16)(data_end - data);\n\n    // Initialize our last byte of data.\n    __u8* last = data + off;\n\n    // Make sure our last byte is within the packet's bounds by comparing to data and data_end.\n    if (last + 1 \u003e (__u8*)data_end)\n        return XDP_PASS;\n\n    if (last \u003c (__u8*)data)\n        return XDP_PASS;\n\n    // Print the last byte of data to /sys/kernel/tracing/trace_pipe.\n    bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n\n    return XDP_PASS;\n}\n\nchar _license[] SEC(\"license\") = \"GPL\";\n```\n\nThis fails with the following.\n\n```\nlibbpf: prog 'prog': BPF program load failed: Permission denied\nlibbpf: prog 'prog': -- BEGIN PROG LOAD LOG --\n0: R1=ctx(off=0,imm=0) R10=fp0\n; void* data = (void*)(long)ctx-\u003edata;\n0: (61) r2 = *(u32 *)(r1 +0)          ; R1=ctx(off=0,imm=0) R2_w=pkt(off=0,r=0,imm=0)\n; void* data_end = (void*)(long)ctx-\u003edata_end;\n1: (61) r3 = *(u32 *)(r1 +4)          ; R1=ctx(off=0,imm=0) R3_w=pkt_end(off=0,imm=0)\n; __u16 off = (__u16)(data_end - data);\n2: (bf) r4 = r3                       ; R3_w=pkt_end(off=0,imm=0) R4_w=pkt_end(off=0,imm=0)\n3: (1f) r4 -= r2                      ; R2_w=pkt(off=0,r=0,imm=0) R4_w=scalar()\n; __u8* last = data + off;\n4: (57) r4 \u0026= 65535                   ; R4_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n; __u8* last = data + off;\n5: (bf) r1 = r2                       ; R1_w=pkt(off=0,r=0,imm=0) R2_w=pkt(off=0,r=0,imm=0)\n6: (0f) r1 += r4                      ; R1_w=pkt(id=1,off=0,r=0,umax=65535,var_off=(0x0; 0xffff)) R4_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n; if (last + 1 \u003e (__u8*)data_end)\n7: (bf) r4 = r1                       ; R1_w=pkt(id=1,off=0,r=0,umax=65535,var_off=(0x0; 0xffff)) R4_w=pkt(id=1,off=0,r=0,umax=65535,var_off=(0x0; 0xffff))\n8: (07) r4 += 1                       ; R4_w=pkt(id=1,off=1,r=0,umax=65535,var_off=(0x0; 0xffff))\n; if (last + 1 \u003e (__u8*)data_end)\n9: (2d) if r4 \u003e r3 goto pc+6          ; R3_w=pkt_end(off=0,imm=0) R4_w=pkt(id=1,off=1,r=0,umax=65535,var_off=(0x0; 0xffff))\n10: (2d) if r2 \u003e r1 goto pc+5         ; R1_w=pkt(id=1,off=0,r=0,umax=65535,var_off=(0x0; 0xffff)) R2_w=pkt(off=0,r=0,imm=0)\n; bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n11: (71) r3 = *(u8 *)(r1 +0)\ninvalid access to packet, off=0 size=1, R1(id=1,off=0,r=0)\nR1 offset is outside of the packet\nprocessed 12 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0\n-- END PROG LOAD LOG --\nlibbpf: prog 'prog': failed to load: -13\nlibbpf: failed to load object 'build/last_one.o'\n```\n\n**Note** - Even checking the value of `off` to make sure it's over 0 doesn't work (e.g. `if (off \u003c 1) return XDP_PASS;`).\n\n### `last_two.c`\nIn our [last_two.c](./src/last_two.c) XDP program, we try a different approach by initializing and checking both the IPv4 and TCP headers. We are only dealing with TCP packets in this case. We then retrieve the total length of the IP header by accessing the `iph-\u003etotal_len` field and then converting the value from *network byte order* to *host byte order* since it's an 16-bit integer.\n\nwe also initialize a pointer to the start of our packet's payload.\n\nFrom there, we retrieve our offset to the the packet's last byte from the start of our payload data by subtracting the size of our IP and TCP headers from the IP header's total length. Aftewards, we initialize the last byte of data by adding the start of our payload data in memory to our offset calculated above and check if the last byte is within the packet's bounds by comparing to `data` and `data_end`.\n\n```C\n#include \u003clinux/bpf.h\u003e\n\n#include \u003clinux/if_ether.h\u003e\n#include \u003clinux/ip.h\u003e\n#include \u003clinux/tcp.h\u003e\n\n#include \u003cnetinet/in.h\u003e\n\n#include \"common.h\"\n\nSEC(\"xdp_prog\")\nint prog(struct xdp_md* ctx) {\n    // Initialize data and data end.\n    void* data = (void*)(long)ctx-\u003edata;\n    void* data_end = (void*)(long)ctx-\u003edata_end;\n\n    // Initialite IP header and check.\n    struct iphdr* iph = data + sizeof(struct ethhdr);\n\n    if (iph + 1 \u003e (struct iphdr *)data_end)\n        return XDP_DROP;\n\n    // We only want to deal with TCP packets.\n    if (iph-\u003eprotocol != IPPROTO_TCP)\n        return XDP_PASS;\n\n    // Initialize TCP header and check.\n    struct tcphdr* tcph = data + sizeof(struct ethhdr) + (iph-\u003eihl * 4);\n\n    if (tcph + 1 \u003e (struct tcphdr *)data_end)\n        return XDP_DROP;\n\n    // Retrieve IP header's length.\n    __u16 ipLen = ntohs(iph-\u003etot_len);\n\n    // Initialize payload.\n    __u8* pl = data + sizeof(struct ethhdr) + (iph-\u003eihl * 4) + (tcph-\u003edoff * 4);\n    \n    // Retrieve offset to last packet.\n    __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n\n    // Initialize last byte of data and check.\n    __u8* last = pl + off;\n\n    if (last + 1 \u003e (__u8*)data_end)\n        return XDP_PASS;\n\n    if (last \u003c (__u8*)data)\n        return XDP_PASS;\n\n    // Print the last byte of data to /sys/kernel/tracing/trace_pipe.\n    bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n\n    return XDP_PASS;\n}\n\nchar _license[] SEC(\"license\") = \"GPL\";\n```\n\nThis also fails with the following.\n\n```\nlibbpf: prog 'prog': BPF program load failed: Permission denied\nlibbpf: prog 'prog': -- BEGIN PROG LOAD LOG --\n0: R1=ctx(off=0,imm=0) R10=fp0\n; int prog(struct xdp_md *ctx) {\n0: (b7) r0 = 1                        ; R0_w=1\n; void* data_end = (void*)(long)ctx-\u003edata_end;\n1: (61) r2 = *(u32 *)(r1 +4)          ; R1=ctx(off=0,imm=0) R2_w=pkt_end(off=0,imm=0)\n; void* data = (void*)(long)ctx-\u003edata;\n2: (61) r1 = *(u32 *)(r1 +0)          ; R1_w=pkt(off=0,r=0,imm=0)\n; if (iph + 1 \u003e (struct iphdr *)data_end)\n3: (bf) r3 = r1                       ; R1_w=pkt(off=0,r=0,imm=0) R3_w=pkt(off=0,r=0,imm=0)\n4: (07) r3 += 34                      ; R3_w=pkt(off=34,r=0,imm=0)\n; if (iph + 1 \u003e (struct iphdr *)data_end)\n5: (2d) if r3 \u003e r2 goto pc+31         ; R2_w=pkt_end(off=0,imm=0) R3_w=pkt(off=34,r=34,imm=0)\n; if (iph-\u003eprotocol != IPPROTO_TCP)\n6: (71) r3 = *(u8 *)(r1 +23)          ; R1_w=pkt(off=0,r=34,imm=0) R3_w=scalar(umax=255,var_off=(0x0; 0xff))\n7: (b7) r0 = 2                        ; R0_w=2\n; if (iph-\u003eprotocol != IPPROTO_TCP)\n8: (55) if r3 != 0x6 goto pc+28       ; R3_w=6\n; if (tcph + 1 \u003e (struct tcphdr *)data_end)\n9: (bf) r3 = r1                       ; R1_w=pkt(off=0,r=34,imm=0) R3_w=pkt(off=0,r=34,imm=0)\n10: (07) r3 += 54                     ; R3_w=pkt(off=54,r=34,imm=0)\n11: (b7) r0 = 1                       ; R0=1\n; if (tcph + 1 \u003e (struct tcphdr *)data_end)\n12: (2d) if r3 \u003e r2 goto pc+24        ; R2=pkt_end(off=0,imm=0) R3=pkt(off=54,r=54,imm=0)\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n13: (71) r4 = *(u8 *)(r1 +14)         ; R1=pkt(off=0,r=54,imm=0) R4_w=scalar(umax=255,var_off=(0x0; 0xff))\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n14: (67) r4 \u003c\u003c= 2                     ; R4_w=scalar(umax=1020,var_off=(0x0; 0x3fc))\n15: (57) r4 \u0026= 60                     ; R4_w=scalar(umax=60,var_off=(0x0; 0x3c))\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n16: (69) r5 = *(u16 *)(r1 +46)        ; R1=pkt(off=0,r=54,imm=0) R5_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n17: (77) r5 \u003e\u003e= 2                     ; R5_w=scalar(umax=16383,var_off=(0x0; 0x3fff))\n18: (57) r5 \u0026= 60                     ; R5_w=scalar(umax=60,var_off=(0x0; 0x3c))\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n19: (0f) r5 += r4                     ; R4_w=scalar(umax=60,var_off=(0x0; 0x3c)) R5_w=scalar(umax=120,var_off=(0x0; 0x7c))\n; __u16 ipLen = ntohs(iph-\u003etot_len);\n20: (69) r4 = *(u16 *)(r1 +16)        ; R1=pkt(off=0,r=54,imm=0) R4_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n21: (dc) r4 = be16 r4                 ; R4_w=scalar()\n; __u16 off = ipLen - (iph-\u003eihl * 4) - (tcph-\u003edoff * 4);\n22: (1f) r4 -= r5                     ; R4_w=scalar() R5_w=scalar(umax=120,var_off=(0x0; 0x7c))\n23: (57) r4 \u0026= 65535                  ; R4_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n; __u8 *last = pl + off;\n24: (0f) r3 += r4                     ; R3_w=pkt(id=1,off=54,r=0,umax=65535,var_off=(0x0; 0xffff)) R4_w=scalar(umax=65535,var_off=(0x0; 0xffff))\n; if (last + 1 \u003e (__u8 *)data_end)\n25: (bf) r4 = r3                      ; R3_w=pkt(id=1,off=54,r=0,umax=65535,var_off=(0x0; 0xffff)) R4_w=pkt(id=1,off=54,r=0,umax=65535,var_off=(0x0; 0xffff))\n26: (07) r4 += 1                      ; R4_w=pkt(id=1,off=55,r=0,umax=65535,var_off=(0x0; 0xffff))\n27: (b7) r0 = 2                       ; R0_w=2\n; if (last + 1 \u003e (__u8 *)data_end)\n28: (2d) if r4 \u003e r2 goto pc+8         ; R2=pkt_end(off=0,imm=0) R4_w=pkt(id=1,off=55,r=0,umax=65535,var_off=(0x0; 0xffff))\n29: (b7) r0 = 2                       ; R0=2\n30: (2d) if r1 \u003e r3 goto pc+6         ; R1=pkt(off=0,r=54,imm=0) R3=pkt(id=1,off=54,r=0,umax=65535,var_off=(0x0; 0xffff))\n; bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n31: (71) r3 = *(u8 *)(r3 +0)\ninvalid access to packet, off=54 size=1, R3(id=1,off=54,r=0)\nR3 offset is outside of the packet\nprocessed 32 insns (limit 1000000) max_states_per_insn 0 total_states 2 peak_states 2 mark_read 1\n-- END PROG LOAD LOG --\nlibbpf: prog 'prog': failed to load: -13\nlibbpf: failed to load object 'build/last_two.o'\n```\n\n### `last_three.c`\nIn our [last_three.c](./src/last_three.c) XDP program, we try to retrieve the last byte of data using the `data_end` pointer which is supposed to represent the location in memory where the packet ends (should be the last byte of the packet, though, this doesn't guarantee it's the last byte of the packet's payload due to the possibility of extra padding, etc. being added). Unfortunately, using `data_end` is prohibited in XDP.\n\n```C\n#include \u003clinux/bpf.h\u003e\n\n#include \"common.h\"\n\nSEC(\"xdp_prog\")\nint prog(struct xdp_md* ctx) {\n    // Initialize data and data_end.\n    void* data = (void*)(long)ctx-\u003edata;\n    void* data_end = (void*)(long)ctx-\u003edata_end;\n\n    // Initialize our last byte of data.\n    __u8* last = data_end;\n\n    // Make sure our last byte is within the packet's bounds by comparing to data and data_end.\n    if (last \u003e (__u8*)data_end)\n        return XDP_PASS;\n\n    if (last \u003c (__u8*)data)\n        return XDP_PASS;\n\n    // Print the last byte of data to /sys/kernel/tracing/trace_pipe.\n    bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n\n    return XDP_PASS;\n}\n\nchar _license[] SEC(\"license\") = \"GPL\";\n```\n\nThis results in the following error.\n\n```\nlibbpf: prog 'prog': BPF program load failed: Permission denied\nlibbpf: prog 'prog': -- BEGIN PROG LOAD LOG --\n0: R1=ctx(off=0,imm=0) R10=fp0\n; void* data_end = (void*)(long)ctx-\u003edata_end;\n0: (61) r2 = *(u32 *)(r1 +4)          ; R1=ctx(off=0,imm=0) R2_w=pkt_end(off=0,imm=0)\n; void* data = (void*)(long)ctx-\u003edata;\n1: (61) r1 = *(u32 *)(r1 +0)          ; R1_w=pkt(off=0,r=0,imm=0)\n; if (last \u003c (__u8*)data)\n2: (2d) if r1 \u003e r2 goto pc+5          ; R1_w=pkt(off=0,r=0,imm=0) R2_w=pkt_end(off=0,imm=0)\n; bpf_printk(\"Last byte of packet is %d.\\n\", *last);\n3: (71) r3 = *(u8 *)(r2 +0)\nR2 invalid mem access 'pkt_end'\nprocessed 4 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0\n-- END PROG LOAD LOG --\nlibbpf: prog 'prog': failed to load: -13\nlibbpf: failed to load object 'build/last_three.o'\n```\n\n## Conclusion\nThis is unfortunately not yet resolved in my case, but I will update this repository when/if I do find a solution. I really feel our [last_one.c](./src/last_one.c) XDP program is the best option due to its simplicity and not relying on the value of `iph-\u003etotal_len` (since this can be set incorrectly in malformed packets).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgamemann%2Fxdp-access-last-byte","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgamemann%2Fxdp-access-last-byte","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgamemann%2Fxdp-access-last-byte/lists"}