{"id":16895731,"url":"https://github.com/ytti/packet_via_dmem","last_synced_at":"2025-06-16T05:06:06.769Z","repository":{"id":27886312,"uuid":"31377719","full_name":"ytti/packet_via_dmem","owner":"ytti","description":"generate pcap from junos trio MQ/LU capture","archived":false,"fork":false,"pushed_at":"2018-07-18T18:30:04.000Z","size":50,"stargazers_count":8,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-25T10:22:28.098Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ytti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-02-26T16:57:11.000Z","updated_at":"2024-08-27T04:16:46.000Z","dependencies_parsed_at":"2022-07-08T10:51:33.131Z","dependency_job_id":null,"html_url":"https://github.com/ytti/packet_via_dmem","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ytti%2Fpacket_via_dmem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ytti%2Fpacket_via_dmem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ytti%2Fpacket_via_dmem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ytti%2Fpacket_via_dmem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ytti","download_url":"https://codeload.github.com/ytti/packet_via_dmem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248347701,"owners_count":21088720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T17:25:45.663Z","updated_at":"2025-04-11T05:32:56.382Z","avatar_url":"https://github.com/ytti.png","language":"Ruby","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Packet Via DMEM\nFinds junos packet-via-dmem packets from arbitrary output and generates text2pcap compatible output\n\n## JunOS\nTo capture say packets with IP address 10.11.12.13\n\n    % ssh test2nqe31.dk|tee output.txt\n    fisakytt@test2nqe31-re1.dk\u003e start shell pfe network afeb0\n\n    AFEB platform (1000Mhz QorIQ P2020 processor, 2048MB memory, 512KB flash)\n    MX104-ABB-0(test2nqe31-re1.dk vty)# test jnh 0 packet-via-dmem enable\n    MX104-ABB-0(test2nqe31-re1.dk vty)# test jnh 0 packet-via-dmem capture 0x3 0x0a0b0c0d\n    MX104-ABB-0(test2nqe31-re1.dk vty)# test jnh 0 packet-via-dmem capture 0x0\n    MX104-ABB-0(test2nqe31-re1.dk vty)# test jnh 0 packet-via-dmem dump\n    MX104-ABB-0(test2nqe31-re1.dk vty)# test jnh 0 packet-via-dmem disable\n\n * in capture the 0x3 sets flags for what type of packets we want, for 0x3 we\n   set two flags on, 'Packet' and 'PacketHead' (show mqchip N lo stats). These\n   two cover all real traffic.\n   1. Packet     (whole packet seen)\n   2. PacketHead (334B packet, inclusive internal headers)\n\n * after capture flags we can set up-to 8 bytes of data to match anywhere in\n   32B window\n\n * after match trigger we have optional 3rd argument which gives the byte\n   offset where our 32B window starts from, default to 0. If you're capturing\n   IPv6, the DADDR won't fit in the first 32B window, so you might give offest\n   of say 19B to get DADDR there too (in case of L2 MAC headers, without VLAN\n   or MPLS)\n\n## Install\n    % gem install packet_via_dmem\n\n## CLI\n    % ./bin/packet-via-dmem --both ~/output.txt|grep Packet|wc -l\n    55\n    % ./bin/packet-via-dmem ~/output.txt|grep Packet|wc -l\n    28\n    % ./bin/packet-via-dmem ~/output.txt|text2pcap - output.pcap\n    Input from: Standard input\n    Output to: output.pcap\n    Output format: PCAP\n    Wrote packet of 66 bytes.\n    Wrote packet of 70 bytes.\n    Wrote packet of 70 bytes.\n    Wrote packet of 246 bytes.\n    Wrote packet of 256 bytes.\n    Wrote packet of 135 bytes.\n    Wrote packet of 246 bytes.\n    Wrote packet of 135 bytes.\n    Wrote packet of 57 bytes.\n    Wrote packet of 57 bytes.\n    Wrote packet of 72 bytes.\n    Wrote packet of 135 bytes.\n    Wrote packet of 256 bytes.\n    Wrote packet of 186 bytes.\n    Wrote packet of 225 bytes.\n    Wrote packet of 66 bytes.\n    Wrote packet of 256 bytes.\n    Wrote packet of 135 bytes.\n    Wrote packet of 142 bytes.\n    Wrote packet of 57 bytes.\n    Wrote packet of 256 bytes.\n    Wrote packet of 186 bytes.\n    Wrote packet of 246 bytes.\n    Wrote packet of 100 bytes.\n    Wrote packet of 57 bytes.\n    Wrote packet of 128 bytes.\n    Wrote packet of 66 bytes.\n    Wrote packet of 9 bytes.\n    Read 28 potential packets, wrote 28 packets (4388 bytes).\n    % ./bin/packet-via-dmem --help\n    usage: ./bin/packet-via-dmem [options]\n        --headers       print headers to stderr\n        -o, --original  print original frames\n        -r, --received  print received frames only (DEFAULT)\n        -s, --sent      print sent frames only\n        -b, --both      print received and sent frames\n        --poprx         pop N bytes from received frames\n        --poptx         pop N bytes from sent frames\n        -d, --debug     turn on debugging\n        -h, --help\n    %\n\nYou can also read from STDIN for live capture, some thing like this should work:\n\n    % sshfs nms: nms\n    % tail -fn 10000 nms/output.txt|packet-via-dmem -|text2pcap - -|wireshark -k -i -\n\n## Library\n    require 'packet_via_dmem'\n    dmem = PacketViaDMEM.new\n    packets = dmem.parse File.read(ARGF[0])\n    packets.each do |capture|\n      p capture.type\n      p capture.packet\n      p capture.header\n      p capture.original\n    end\n\n\n## Header format\n### Received header\n\n  * first four bits is message type\n    * 0 'lu packet' (i.e. whole packet was sent for lookup, i.e. small packet)\n    * 1 'lu packet head' (i.e. only head of packet was sent for lookup, i.e. large packet)\n  * next 13 bits is table entry (memory loc)\n  * next 11 bits is stream\n  * next 3 bits offset (for fabric cell sharing)\n  * next bit is size (17th bit, no way to test, would need \u003e65k packets)\n  * next 16 bits is size (if message type is 1)\n  * next 8 bits is port\n  * next 8 bits is packet type\n\n  * packet type\n    * 0x00 real PITA, looks to be packets from control-plane, but amount of bytes that I need to pop I can't really figure out. I now rely on port# which likely isn't robust. This made me change my mind that 5th byte isn't port, but combined type, as it allowed, what seemed cleaner classification, but unfortunately it is not so.\n    * 0x08 is no-op, i don't need to pop anything, DMAC follows\n    * 0x20 this seems to be quite reliably mpls, i need to do some magic, as we're missing ethertype and my macs are wrong, there is also two extra bytes\n    * 0x80 is some short trash (payload 0xc013c6752759644ae0) no clue what it is\n\nExample from MX960\n\n    00 00 c0 30 80 08\n    00 03 40 30 80 08\n    00 03 c0 70 81 08\n    00 06 c0 30 80 08\n    00 07 c0 70 81 08\n    00 0a 40 30 80 08\n    00 01 c0 70 81 08\n    00 02 40 70 81 08\n    10 01 40 70 05 c0 81 08\n    00 05 40 70 81 08\n    00 08 c0 30 80 08\n    00 0a c0 70 81 08\n    10 0b 40 20 05 28 40 08\n    00 0d c0 30 80 08\n    10 00 c0 30 05 8c 80 08\n    00 03 c0 30 80 08\n    10 03 40 30 05 8c 80 08\n    10 06 40 30 05 8c 80 08\n    10 06 c0 30 05 f0 80 08\n    00 07 40 70 81 08\n    00 07 80 40 42 20\n    00 09 00 98 42 20\n    00 0a 00 48 42 20\n    10 09 c0 30 05 8c 80 08\n    00 02 40 70 81 08\n    10 0b 80 48 05 ce 42 20\n    10 01 c0 30 05 8c 80 08\n\nExample from MX480\n\n    00 0b 40 60 41 08\n    00 01 c0 70 81 08\n    00 02 40 70 81 08\n    00 02 c0 70 81 08\n    10 03 40 70 05 40 81 08\n    00 03 c0 70 81 08\n    00 06 40 70 81 08\n    00 07 c0 70 81 08\n    00 08 47 f0 20 00\n    00 09 45 f0 20 00\n    00 09 c7 f0 80 00\n    00 0b c0 70 81 08\n    10 0c 08 00 02 00 1f 00\n    00 00 c0 70 81 08\n    00 01 47 f0 80 00\n    00 04 40 60 41 08\n    10 04 c0 70 01 50 81 08\n    00 05 40 70 81 08\n    00 05 c0 70 81 08\n    00 06 c5 f0 20 00\n    10 07 08 00 02 00 1f 00\n    00 08 c0 70 81 08\n    00 0a 40 70 81 08\n    00 0a c0 70 81 08\n    00 0b 47 f0 20 00\n    00 01 c0 70 81 08\n    00 02 40 60 41 08\n    00 07 c7 f0 b0 80\n\nExample from MX80\n\n    00 08 00 f0 81 08\n    10 08 80 f0 05 b4 81 08\n    10 09 00 f0 05 b4 81 08\n    10 09 80 f0 05 b4 81 08\n    00 0a 00 f0 92 08\n    10 0a 80 f0 05 b4 81 08\n    10 03 00 f0 05 b4 81 08\n    00 04 00 f0 81 08\n    10 04 80 f0 05 b4 81 08\n    10 05 00 f0 05 b4 81 08\n    10 05 80 f0 05 b4 81 08\n    00 06 00 f0 81 08\n    10 06 80 f0 05 b4 81 08\n    00 07 00 f0 80 08\n    10 07 80 f0 05 b4 81 08\n    10 0b 00 f0 02 28 81 08\n\n### Sent header\n  *FIXME* check the source...\n\nExample from MX960\n\n    00 bf e0 0d 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 07 e9\n    00 bf e0 0f 71 f0 00 09 42 20 01 44 03 01 01 21 00 00 00 00 00 00 16 65\n    00 bf e0 14 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 0b ad\n    00 bf e0 03 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 04 06\n    00 bf e0 04 71 f0 00 00 42 20 01 44 03 01 00 01 00 00 00 00 00 00 24 42\n    00 bf e0 0a 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 18 a4\n    00 a0 00 02 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 0a 8f\n    00 bf e0 11 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 04 00\n    00 bf e0 15 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 1c 69\n    00 bf e0 1b 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 0b ad\n    00 a0 00 16 71 f0 00 04 42 20 01 44 03 01 00 81 00 00 00 00 00 00 05 ec\n    00 bf e0 07 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 0b ad\n    00 a0 00 01 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 08 0a\n    00 a0 00 06 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 08 0a\n    00 a0 00 0c 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 08 0a\n    00 a0 00 0d 71 f0 00 00 42 20 01 44 03 01 00 01 00 00 00 00 00 00 24 06\n    08 bf e0 0f 70 00 00 08 b0 0e 80 03 0a\n    00 bf e0 0e 11 f0 00 04 42 20 01 44 00 01 00 81 00 00 00 00 00 00 06 6b\n    08 bf e0 12 10 00 00 08 b0 0e 80 03 0a\n    08 bf e0 14 10 00 00 08 b0 0e 80 03 0a\n    00 bf e0 04 71 f0 00 09 42 20 01 44 03 01 01 21 00 00 00 00 00 00 16 65\n\nExample from MX480\n\n    00 bf e0 16 10 00 03 f9 20 00 20 03 02 b0 03 7a 00 0e 00 42 80 00 00 20 0e 00 00 10 00 0c 00 00 00\n    00 bf e0 03 10 00 03 f8 20 40 20 00 20 10 03 7a 00 12 00 46 80 00 00 20 12 00 00 18 00 00 00 00 00\n    00 bf e0 04 10 00 03 f8 20 40 20 00 20 10 03 7a 00 12 00 46 80 00 00 20 12 00 00 18 00 00 00 00 00\n    08 bf e0 05 14 00 00 10 20 12 80 5a 28\n    08 a0 00 06 14 00 00 10 b0 12 80 5a 28\n    08 bf e0 07 14 00 00 10 20 12 80 5a 28\n    08 bf e0 0c 14 00 00 10 b0 12 80 5a 28\n    08 bf e0 0f 14 00 00 10 20 12 80 5a 28\n    08 bf e0 10 14 00 00 0b 20 12 80 33 2a\n    08 bf e0 12 14 00 00 0b 20 0e 80 33 2c\n    08 bf e0 13 14 00 00 08 00 00 80 00 be\n    08 bf e0 17 14 00 00 10 20 12 80 5a 28\n    08 bf e0 01 14 00 00 10 b0 12 80 5a 28\n    08 bf e0 02 14 00 00 08 00 00 80 00 be\n    08 a0 00 09 14 00 00 10 a0 12 80 5a 28\n    08 bf e0 0a 14 00 00 10 20 12 80 5a 28\n    08 bf e0 0b 14 00 00 10 a0 12 80 5a 28\n    08 bf e0 0d 14 00 00 0b 00 0e 80 33 2c\n    08 bf e0 11 14 00 00 10 b0 12 80 5a 28\n    08 bf e0 14 14 00 00 10 b0 12 80 5a 28\n    08 bf e0 15 14 00 00 10 20 12 80 5a 28\n    08 bf e0 16 14 00 00 0b 20 0e 80 33 2c\n\nExample from MX80\n\n    08 bf e0 10 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 11 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 12 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 13 11 00 00 00 10 0e 80 0a 1e\n    08 bf e0 14 11 00 00 00 70 12 80 0a 1e\n    08 a0 00 15 11 00 00 00 70 0e 80 0a 1e\n    08 bf e0 08 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 06 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 09 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 0a 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 0b 11 00 00 00 70 0e 80 0a 1e\n    08 bf e0 0c 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 0d 11 00 00 00 70 0e 80 0a 1e\n    08 bf e0 0e 71 00 00 08 10 0e 80 0a 32\n    08 a0 00 0f 11 00 00 00 70 0e 80 0a 1e\n    08 a0 00 16 11 00 00 00 70 0e 80 0a 1e\n\n## Todo\n  1. reverse engineer sent headers (so we can pop them correctly)\n  1. reverse engineer cookie\n  1. more research on received headers source fabric, port, npu\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fytti%2Fpacket_via_dmem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fytti%2Fpacket_via_dmem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fytti%2Fpacket_via_dmem/lists"}