{"id":33003035,"url":"https://github.com/catid/wirehair","last_synced_at":"2026-02-09T16:15:39.093Z","repository":{"id":2374402,"uuid":"3339228","full_name":"catid/wirehair","owner":"catid","description":"Wirehair : O(N) Fountain Code for Large Data","archived":false,"fork":false,"pushed_at":"2023-12-02T22:20:06.000Z","size":1719,"stargazers_count":287,"open_issues_count":8,"forks_count":60,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-11-18T08:08:54.190Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"http://wirehairfec.com","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/catid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2012-02-02T21:55:49.000Z","updated_at":"2025-11-06T07:47:00.000Z","dependencies_parsed_at":"2023-07-07T20:16:45.579Z","dependency_job_id":null,"html_url":"https://github.com/catid/wirehair","commit_stats":{"total_commits":262,"total_committers":10,"mean_commits":26.2,"dds":0.4961832061068703,"last_synced_commit":"cfd0b2ec831f3503448390c89ce6d94fc74532d1"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/catid/wirehair","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catid%2Fwirehair","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catid%2Fwirehair/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catid%2Fwirehair/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catid%2Fwirehair/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/catid","download_url":"https://codeload.github.com/catid/wirehair/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/catid%2Fwirehair/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29272090,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-09T13:47:44.167Z","status":"ssl_error","status_checked_at":"2026-02-09T13:47:43.721Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-11-13T14:00:38.909Z","updated_at":"2026-02-09T16:15:39.073Z","avatar_url":"https://github.com/catid.png","language":"C++","funding_links":[],"categories":["Serialization"],"sub_categories":[],"readme":"# Wirehair\n## Fast and Portable Fountain Codes in C\n\nWirehair produces a stream of error correction blocks from a data source\nusing an erasure code.  When enough of these blocks are received,\nthe original data can be recovered.\n\nAs compared to other similar libraries, an unlimited number of error\ncorrection blocks can be produced, and much larger block counts are supported.\nFurthermore, it gets slower as O(N) in the amount of input data rather\nthan O(N Log N) like the Leopard block code or O(N^2) like the Fecal fountain code,\nso it is well-suited for large data.\n\nThis is not an ideal MDS code, so sometimes it will fail to recover N\noriginal data packets from N symbol packets.  It may take N + 1 or N + 2 or more.\nOn average it takes about N + 0.02 packets to recover.  Overall the overhead\nfrom the code inefficiency is low, compared to LDPC and many other fountain codes.\n\nA simple C API is provided to make it easy to incorporate into existing\nprojects.  No external dependencies are required.\n\n\n##### Building: Quick Setup\n\nThe source code in this folder (gf256 and wirehair code) can be incorporated\ninto your project without any other external dependencies.\n\nTo build the software in this repo:\n\nOn Windows, make sure CMake and Git Bash are installed.  Open up git bash and then:\n\n~~~\ngit clone git@github.com:catid/wirehair.git\ncd wirehair\nmkdir build\ncd build\ncmake .. -G \"Visual Studio 16 2019\"\nls\nexplorer .\n~~~\n\nThen you can use Visual Studio Community Edition to open up the `wirehair.sln` file and build the software.\n\n\n#### Example Usage\n\nHere's an example program using Wirehair.  It's included in the UnitTest project and demonstrates both the sender and receiver, which are normally separate programs.  For example the data sender might be a file server and the data receiver might be downloading a file from the sender.\n\n~~~\n#include \u003cwirehair/wirehair.h\u003e\n\nstatic bool ReadmeExample()\n{\n    // Size of packets to produce\n    static const int kPacketSize = 1400;\n\n    // Note: Does not need to be an even multiple of packet size or 16 etc\n    static const int kMessageBytes = 1000 * 1000 + 333;\n\n    vector\u003cuint8_t\u003e message(kMessageBytes);\n\n    // Fill message contents\n    memset(\u0026message[0], 1, message.size());\n\n    // Create encoder\n    WirehairCodec encoder = wirehair_encoder_create(nullptr, \u0026message[0], kMessageBytes, kPacketSize);\n    if (!encoder)\n    {\n        cout \u003c\u003c \"!!! Failed to create encoder\" \u003c\u003c endl;\n        return false;\n    }\n\n    // Create decoder\n    WirehairCodec decoder = wirehair_decoder_create(nullptr, kMessageBytes, kPacketSize);\n    if (!decoder)\n    {\n        // Free memory for encoder\n        wirehair_free(encoder);\n\n        cout \u003c\u003c \"!!! Failed to create decoder\" \u003c\u003c endl;\n        return false;\n    }\n\n    unsigned blockId = 0, needed = 0;\n\n    for (;;)\n    {\n        // Select which block to encode.\n        // Note: First N blocks are the original data, so it's possible to start\n        // sending data while wirehair_encoder_create() is getting started.\n        blockId++;\n\n        // Simulate 10% packetloss\n        if (blockId % 10 == 0) {\n            continue;\n        }\n\n        // Keep track of how many pieces were needed\n        ++needed;\n\n        vector\u003cuint8_t\u003e block(kPacketSize);\n\n        // Encode a packet\n        uint32_t writeLen = 0;\n        WirehairResult encodeResult = wirehair_encode(\n            encoder, // Encoder object\n            blockId, // ID of block to generate\n            \u0026block[0], // Output buffer\n            kPacketSize, // Output buffer size\n            \u0026writeLen); // Returned block length\n\n        if (encodeResult != Wirehair_Success)\n        {\n            cout \u003c\u003c \"wirehair_encode failed: \" \u003c\u003c encodeResult \u003c\u003c endl;\n            return false;\n        }\n\n        // Attempt decode\n        WirehairResult decodeResult = wirehair_decode(\n            decoder, // Decoder object\n            blockId, // ID of block that was encoded\n            \u0026block[0], // Input block\n            writeLen); // Block length\n\n        // If decoder returns success:\n        if (decodeResult == Wirehair_Success) {\n            // Decoder has enough data to recover now\n            break;\n        }\n\n        if (decodeResult != Wirehair_NeedMore)\n        {\n            cout \u003c\u003c \"wirehair_decode failed: \" \u003c\u003c decodeResult \u003c\u003c endl;\n            return false;\n        }\n    }\n\n    vector\u003cuint8_t\u003e decoded(kMessageBytes);\n\n    // Recover original data on decoder side\n    WirehairResult decodeResult = wirehair_recover(\n        decoder,\n        \u0026decoded[0],\n        kMessageBytes);\n\n    if (decodeResult != Wirehair_Success)\n    {\n        cout \u003c\u003c \"wirehair_recover failed: \" \u003c\u003c decodeResult \u003c\u003c endl;\n        return false;\n    }\n\n    // Free memory for encoder and decoder\n    wirehair_free(encoder);\n    wirehair_free(decoder);\n\n    return true;\n}\n\nint main()\n{\n    const WirehairResult initResult = wirehair_init();\n\n    if (initResult != Wirehair_Success)\n    {\n        SIAMESE_DEBUG_BREAK();\n        cout \u003c\u003c \"!!! Wirehair initialization failed: \" \u003c\u003c initResult \u003c\u003c endl;\n        return -1;\n    }\n\n    if (!ReadmeExample())\n    {\n        SIAMESE_DEBUG_BREAK();\n        cout \u003c\u003c \"!!! Example usage failed\" \u003c\u003c endl;\n        return -2;\n    }\n...\n~~~\n\n\n#### Benchmarks\n\nSome quick comments:\n\nBenchmarks on my PC do not mean a whole lot.  Right now it's clocked at 3 GHz and has Turbo Boost on, etc.\nTo run the test yourself just build and run the UnitTest project in Release mode.\n\nFor small values of N \u003c 128 or so this is a pretty inefficient codec compared to the Fecal codec.  Fecal is also a fountain code but is limited to repairing a small number of failures or small input block count.\n\n~~~\nFor N = 2 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 11 usec (236.364 MBPS)\n+ Average wirehair_encode() time: 0 usec (7435.7 MBPS)\n+ Average wirehair_decode() time: 2 usec (476.205 MBPS)\n+ Average overhead piece count beyond N = 0.0105\n+ Average wirehair_recover() time: 0 usec (9319 MBPS)\n\nFor N = 4 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 8 usec (650 MBPS)\n+ Average wirehair_encode() time: 0 usec (8353.43 MBPS)\n+ Average wirehair_decode() time: 1 usec (695.102 MBPS)\n+ Average overhead piece count beyond N = 0.0225\n+ Average wirehair_recover() time: 0 usec (11219 MBPS)\n\nFor N = 8 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 13 usec (800 MBPS)\n+ Average wirehair_encode() time: 0 usec (7916.2 MBPS)\n+ Average wirehair_decode() time: 1 usec (704.359 MBPS)\n+ Average overhead piece count beyond N = 0.0045\n+ Average wirehair_recover() time: 1 usec (8973.25 MBPS)\n\nFor N = 16 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 27 usec (770.37 MBPS)\n+ Average wirehair_encode() time: 0 usec (7993.4 MBPS)\n+ Average wirehair_decode() time: 1 usec (707.211 MBPS)\n+ Average overhead piece count beyond N = 0.036\n+ Average wirehair_recover() time: 2 usec (9116.81 MBPS)\n\nFor N = 32 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 41 usec (1014.63 MBPS)\n+ Average wirehair_encode() time: 0 usec (7062.93 MBPS)\n+ Average wirehair_decode() time: 1 usec (908.097 MBPS)\n+ Average overhead piece count beyond N = 0.0195\n+ Average wirehair_recover() time: 5 usec (8057.33 MBPS)\n\nFor N = 64 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 81 usec (1027.16 MBPS)\n+ Average wirehair_encode() time: 0 usec (7159.51 MBPS)\n+ Average wirehair_decode() time: 1 usec (1033.95 MBPS)\n+ Average overhead piece count beyond N = 0.017\n+ Average wirehair_recover() time: 10 usec (7640.74 MBPS)\n\nFor N = 128 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 192 usec (866.667 MBPS)\n+ Average wirehair_encode() time: 0 usec (5662.07 MBPS)\n+ Average wirehair_decode() time: 1 usec (870.14 MBPS)\n+ Average overhead piece count beyond N = 0.015\n+ Average wirehair_recover() time: 25 usec (6419.38 MBPS)\n\nFor N = 256 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 319 usec (1043.26 MBPS)\n+ Average wirehair_encode() time: 0 usec (6333.2 MBPS)\n+ Average wirehair_decode() time: 1 usec (1018.77 MBPS)\n+ Average overhead piece count beyond N = 0.022\n+ Average wirehair_recover() time: 50 usec (6602.26 MBPS)\n\nFor N = 512 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 670 usec (993.433 MBPS)\n+ Average wirehair_encode() time: 0 usec (6483.91 MBPS)\n+ Average wirehair_decode() time: 1 usec (1028.85 MBPS)\n+ Average overhead piece count beyond N = 0.022\n+ Average wirehair_recover() time: 100 usec (6600.1 MBPS)\n\nFor N = 1024 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 1697 usec (784.443 MBPS)\n+ Average wirehair_encode() time: 0 usec (5309.05 MBPS)\n+ Average wirehair_decode() time: 1 usec (671.005 MBPS)\n+ Average overhead piece count beyond N = 0.022\n+ Average wirehair_recover() time: 207 usec (6404.05 MBPS)\n\nFor N = 2048 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 3227 usec (825.039 MBPS)\n+ Average wirehair_encode() time: 0 usec (5202.3 MBPS)\n+ Average wirehair_decode() time: 1 usec (683.141 MBPS)\n+ Average overhead piece count beyond N = 0.021\n+ Average wirehair_recover() time: 441 usec (6026.08 MBPS)\n\nFor N = 4096 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 7614 usec (699.343 MBPS)\n+ Average wirehair_encode() time: 0 usec (4334.08 MBPS)\n+ Average wirehair_decode() time: 2 usec (577.674 MBPS)\n+ Average overhead piece count beyond N = 0.0215\n+ Average wirehair_recover() time: 1208 usec (4405.65 MBPS)\n\nFor N = 8192 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 17208 usec (618.875 MBPS)\n+ Average wirehair_encode() time: 0 usec (3277.17 MBPS)\n+ Average wirehair_decode() time: 2 usec (521.665 MBPS)\n+ Average overhead piece count beyond N = 0.075\n+ Average wirehair_recover() time: 2916 usec (3651.35 MBPS)\n\nFor N = 16384 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 42512 usec (501.016 MBPS)\n+ Average wirehair_encode() time: 0 usec (2646.89 MBPS)\n+ Average wirehair_decode() time: 2 usec (435.173 MBPS)\n+ Average overhead piece count beyond N = 0.015\n+ Average wirehair_recover() time: 7282 usec (2924.63 MBPS)\n\nFor N = 32768 packets of 1300 bytes:\n+ Average wirehair_encoder_create() time: 111287 usec (382.78 MBPS)\n+ Average wirehair_encode() time: 0 usec (2378.29 MBPS)\n+ Average wirehair_decode() time: 3 usec (342.556 MBPS)\n+ Average overhead piece count beyond N = 0.0195\n+ Average wirehair_recover() time: 16326 usec (2609.23 MBPS)\n~~~\n\n\n#### Credits\n\nSoftware by Christopher A. Taylor \u003cmrcatid@gmail.com\u003e\n\nPlease reach out if you need support or would like to collaborate on a project.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatid%2Fwirehair","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcatid%2Fwirehair","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcatid%2Fwirehair/lists"}