{"id":31815929,"url":"https://github.com/xilinx/hls_packet_processing","last_synced_at":"2025-10-11T09:27:03.512Z","repository":{"id":53339680,"uuid":"196094684","full_name":"Xilinx/HLS_packet_processing","owner":"Xilinx","description":null,"archived":false,"fork":false,"pushed_at":"2019-12-10T15:19:08.000Z","size":124,"stargazers_count":40,"open_issues_count":4,"forks_count":21,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-01-02T18:57:17.693Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Xilinx.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-07-09T22:52:23.000Z","updated_at":"2023-12-13T01:52:45.000Z","dependencies_parsed_at":"2022-09-10T10:38:35.156Z","dependency_job_id":null,"html_url":"https://github.com/Xilinx/HLS_packet_processing","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/Xilinx/HLS_packet_processing","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xilinx%2FHLS_packet_processing","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xilinx%2FHLS_packet_processing/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xilinx%2FHLS_packet_processing/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xilinx%2FHLS_packet_processing/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Xilinx","download_url":"https://codeload.github.com/Xilinx/HLS_packet_processing/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Xilinx%2FHLS_packet_processing/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279006760,"owners_count":26084178,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-11T02:00:06.511Z","response_time":55,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-11T09:26:53.083Z","updated_at":"2025-10-11T09:27:03.505Z","avatar_url":"https://github.com/Xilinx.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# HLS Packet Processing\n\nThis repository contains a set of Vivado HLS libraries supporting networking applications.\n\nSeveral applications of the library are provided:\n\n- apps/arp: An ARP client implementation including ARP cache.\n- apps/mold_remover_packet: A parser for MOLD/ITCH messages, using the \"packet-oriented API\".\n- apps/mold_remover_stream: A parser for MOLD/ITCH messages, using the \"stream-oriented API\".\n- apps/traffic_manager: A packet FIFO which preferentially drops low-priority packets.\n- apps/pynq_mqttsn: An implementation of the UDP-based MQTTSN publish/subscribe protocol\n\nThe libraries have several goals in mind:\n- Minimize multiplexing cost for fields with variable offset.\n- Allow parameterized code to deal with different I/O width  (32-512 bits)\n- Capture common header formats in library\n\nConceptually, the approach is inspired by the P4 packet processing language (https://p4.org/),\nbut is embedded in C++ and uses Vivado HLS to generate an FPGA implementation.  This can make verification\nconvient.  A number\nof existing networking designs in Vivado HLS (e.g. https://github.com/Xilinx/HLx_Examples/tree/master/Acceleration/memcached,\nhttps://github.com/fpgasystems/fpga-network-stack) use an RTL-like FSM-oriented coding style.  In constrast,\nthese libraries leverage more capabilities of Vivado HLS to automatically insert pipeline stages as\nappropriate.  These libraries are implemented in C++11 using metaprogramming techniques from the `boost::mpl`\nlibrary.  This approach works with current versions of Vivado HLS (e.g. 2018.3), although it is not yet officially supported.\n\n# Using the library\n\nThe library depends on boost 1.58.0.  If you've checked out the repository, you can check out and build headers for the corresponding version of boost using:\n\n```\ngit submodule update --init --recursive\npushd boost\n./bootstrap.sh\n./b2 headers\npopd\n```\n\nFixed packet headers are represented using `boost::mpl::vector`, and individual fields of the header can be accessed.\n```\nnamespace ipv4 {\n    typedef newfield\u003cap_uint\u003c16\u003e, boost::mpl::string\u003c'sprt'\u003e \u003e sport;\n    typedef newfield\u003cap_uint\u003c16\u003e, boost::mpl::string\u003c'dprt'\u003e \u003e dport;\n    typedef newfield\u003cap_uint\u003c16\u003e, boost::mpl::string\u003c'leng'\u003e \u003e length;\n    typedef newfield\u003cap_uint\u003c16\u003e, boost::mpl::string\u003c'csum'\u003e \u003e checksum; \n\n    using udp_header = fixed_header\u003cboost::mpl::vector\u003csport, dport, length, checksum\u003e \u003e;\n};\n\nudp_header h;\nh.set\u003csport\u003e(5555);\nint i = h.get\u003csport\u003e();\n```\n\n## Packet-oriented coding\nTwo coding styles exist for serializing headers and packets over a stream of arbitrary (power of 2) width.\nThe first style is a **packet-oriented** coding style, where the structure of a packet of interest is\ndefined as a cascade of headers, followed by the payload, represented by the `Packet` class.  An entire\npacket can be sent and received using the `serialize()` and `deserialize()` functions.\n```\n#define BYTESPERCYCLE 8\ntypedef ap_axiu\u003c8*BYTESPERCYCLE,1,1,1\u003e StreamType;\n\nvoid ingress(hls::stream\u003cStreamType\u003e \u0026input,    // input\n             hls::stream\u003cStreamType\u003e \u0026internal,    // output\n             hls::stream\u003cshort\u003e \u0026input_length_stream1, // output\n             hls::stream\u003cshort\u003e \u0026input_length_stream2, // output\n             hls::stream\u003cap_uint\u003c6\u003e \u003e \u0026diffserv_stream // output) {\n    \n    short input_length;\n    ap_uint\u003c6\u003e diffserv;\n\n    Packet p;\n    ipv4_hdr\u003cPacket\u003e ih(p);\n    ethernet_hdr\u003cipv4_hdr\u003cPacket\u003e \u003e eh(ih);\n    eh.deserialize(input);\n    diffserv = ih.diffserv.get() \u003e\u003e 2; // Drop the ECN field.\n    input_length = eh.data_length();\n    eh.serialize(internal);\n```\n\nCode like above generates a store-and-forward architecture in Vivado HLS.  Headers are stored in Flip-Flops\nand can be cheaply accessed in parallel.  Payloads are stored in BRAM/URAM memory and can only be accessed\nsequentially.  With Dataflow mode in Vivado HLS, payload storage can be automatically double-buffered, enabling\nsimultaneous sending and receiving of packets at close to line rate.  Note, however that the end-to-end\nlatency of such a design will typically be more than one packet time.  In addition, the library assumes that\ninput and output packets are always properly framed using TLAST in an AXI stream.  This makes it impossible to\ngenerate an improperly framed output, since the framing is implemented in the library.  Currently the library\nhas no good way of recovering from improperly framed inputs.\n\nAlthough the above example is quite simple, more complex operations on Packets are possible.  In particular,\nheaders can be conceptually added and removed.  Also, Packets can be serialized and deserialized from arrays of bytes.\n```\n// Removing a header\n    Packet p;\n    header\u003cPacket, 40\u003e ih(p);\n    ethernet_hdr\u003cheader\u003cPacket, 40\u003e \u003e eh(ih);\n    eh.deserialize(buf, len);\n    ih.serialize(output, len);\n```\n```\n// Adding a header\n    Packet p;\n    header\u003cPacket, 40\u003e ih(p);\n    ethernet_hdr\u003cheader\u003cPacket, 40\u003e \u003e eh(ih);\n    ih.deserialize(buf, len);\n    eh.serialize(output, len);\n```\nAdditionally, deserialized packets can be reparsed in other formats. Such a reparsing shares the same underlying storage as the original packet, but enables parsing of packets with different header combinations.  The return value is a 'header-like' object which supports the same indexing operations.\n```\n    Packet p;\n    header\u003cPacket, 40\u003e ih(p);\n    ethernet_hdr\u003cheader\u003cPacket, 40\u003e \u003e eh(ih);\n    eh.deserialize(buf, len);\n \n    MACAddressT destinationMAC = eh.get\u003cdestinationMAC\u003e();\n    ap_uint\u003c16\u003e dmp_macType = eh.get\u003cetherType\u003e();\n \n   if (dmp_macType == ethernet::ethernet_etherType::ARP) {\n        stats.arps_received++;\n        auto ah = parse_arp_hdr(ih);\n        do_arp(ah)\n    } else {\n        auto h = ipv4::parse_ipv4_hdr(ih);\n        if(h.get\u003cipv4::protocol\u003e() == ipv4::ipv4_protocol::UDP) {\n            auto uh = ipv4::parse_udp_hdr(h.p);\n            ...\n```\nLastly, we can 'skip' an arbitrary number of bytes in a packet.  Again, this returns a header-like object.   Note, however, that if the offset is not a compile-time constant, then indexing into the resulting object requires reading at a variable address.  This almost inevitably requires a circuit with wide multiplexors.  If significant reading of the fields of a packet after a variable offset are required, then it is usually more hardware efficient to serialize and deserialize the packet, enabling more efficient access.\n```\nPacket p;\nmold_hdr\u003cPacket\u003e mh(p);\n\nint offset = 0;\n\n// Iterate over each mold message, starting at offset 0\n// after the mold header.\nfor(int i = 0; i \u003c mh.get\u003cmessageCount\u003e(); i++) {\n\n   // Extract a mold message from the 'packet' portion\n   auto next_message = skip(p, offset);\n   auto mold_message = parse_mold_message_hdr(next_message);\n\n   // Look at the mold message to figure out where the next one starts.\n   // the length doesn't include the size of the length field itself.\n   offset += mold_message.get\u003cmessageLength\u003e() + 2;\n\n   // Extract the itch message and do something with it.\n   auto itch_message = parse_itch_hdr(mold_message.p);\n```\n\n## Stream-oriented coding style\nAn alternative coding style reads headers one at a time from the input stream.  Normally, this is somewhat awkward to implement, since the core `hls::stream` object is intended to be accessed only a data beat at a time, while headers are rarely the same size as a data beat and rarely aligned.   This coding style is implemented by **reader** and **writer** objects, which encapsulate an `hls::stream` object and provide the additional ability to read or write arbitrary headers of arbitrary size, along with a small amount of storage.\n```\nvoid ingress_buffer_writer(hls::stream\u003cStreamType\u003e \u0026in) {\n    auto reader = make_reader(in);\n    ipv4::header ih;\n    ipv4::udp_header uh;\n    udt::data_header dh;\n    reader.get(ih);\n    reader.get(uh);\n    reader.get(dh);\n\n    bool isValidUDT =  check_headers(ih,uh,dh);\n    if(isValidUDT) {\n        // Copy the packet to the external buffer.\n    write_loop:\n        bool done = false;\n        int inCount = 0;\n        while(!done) {\n#pragma HLS PIPELINE\n            StreamType tmp;\n            reader.read(tmp);\n            if(inCount \u003c MTU+1) {\n                buffer_storage[inCount/BYTESPERCYCLE][buffer_id] = tmp.data;\n                inCount += keptbytes(tmp.keep);\n            }\n            if(tmp.last)\n                done = true;\n        }\n    } else {\n        reader.read_rest();\n    }\n}\n```\nThis coding style naturally leads to cut-through packet processing, where the latency from input to output is approximately one packet.   It also has the advantage that infinite streams of data (i.e. a reconstructed TCP stream) can be parsed, making it better able to handle higher-level (L4-L7) protocols.  The main disadvantage is that it is possible to create incorrectly framed streams (missing the TLAST at end of packet) and that generally speaking, the programmer must be somewhat aware of the state stored in readers and writers.\n\nAs with almost any packet-processing model, headers can be added and removed by simply writing code that sends and receives the correct headers and payload, since there is no concept of a 'packet object'.\n```\n        // Remove header\n        auto reader = make_reader(dataIn);\n        auto writer = make_writer(dataOut);\n        ethernet::header x = reader.get\u003cethernet::header\u003e();\n        writer.put_rest(reader);\n```\n```\n        // Add header\n        auto reader = make_reader(dataIn);\n        auto writer = make_writer(dataOut);\n        ethernet::header x;\n         for(int i = 0; i \u003c LENGTH; i++) {\n            x.set\u003c1\u003e(i, i-(LENGTH));\n        }\n        writer.put(x);\n        writer.put_rest(reader);\n```\n\n# Other libraries\nIn addition to packet parsing, networking application often require a number of other data structures to construct an overall design.  The library contains several support libraries with useful data structures.\n\n### hls::cam\nA Parallel-match CAM.\nUses O(N) LUTs/FFs. II=1 lookup, insert/delete. \n\n### hls::algorithmic_cam\nA Cuckoo-Hashing CAM efficient for large N.\nUses O(N) BRAM bits. ~II=1 lookup, insert/delete (assuming insert and delete are rare operations)\n\n### hls::allocator\nA simple freelist based allocator with II=1 allocate/deallocate\n\n### hls::message_buffer implements \nA bag-like data structure with II=1 insert/pick\n\n\n## License\n\nCopyright (c) 2016-2018, Xilinx, Inc.\nAll rights reserved.\n\nLicensed under the Apache License, Version 2.0 (the \"License\");\nyou may not use this file except in compliance with the License.\nYou may obtain a copy of the License at\n\n    http://www.apache.org/licenses/LICENSE-2.0\n\nUnless required by applicable law or agreed to in writing, software\ndistributed under the License is distributed on an \"AS IS\" BASIS,\nWITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\nSee the License for the specific language governing permissions and\nlimitations under the License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxilinx%2Fhls_packet_processing","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxilinx%2Fhls_packet_processing","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxilinx%2Fhls_packet_processing/lists"}