{"id":16001236,"url":"https://github.com/rj/pickleback","last_synced_at":"2025-03-22T17:47:59.101Z","repository":{"id":209488722,"uuid":"714397319","full_name":"RJ/pickleback","owner":"RJ","description":"A protocol layer for unreliable datagram channels","archived":false,"fork":false,"pushed_at":"2023-12-12T09:28:59.000Z","size":322,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-20T11:08:23.443Z","etag":null,"topics":["gamedev","netcode"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/RJ.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-04T19:35:15.000Z","updated_at":"2023-12-12T05:12:43.000Z","dependencies_parsed_at":"2025-01-27T22:06:15.290Z","dependency_job_id":"0fb357e0-886a-45bb-94ef-c046213b4200","html_url":"https://github.com/RJ/pickleback","commit_stats":null,"previous_names":["rj/packeteer","rj/pickleback"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RJ%2Fpickleback","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RJ%2Fpickleback/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RJ%2Fpickleback/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/RJ%2Fpickleback/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/RJ","download_url":"https://codeload.github.com/RJ/pickleback/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244999230,"owners_count":20544866,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gamedev","netcode"],"created_at":"2024-10-08T09:41:01.125Z","updated_at":"2025-03-22T17:47:59.076Z","avatar_url":"https://github.com/RJ.png","language":"Rust","readme":"# Pickleback\n\nA way to multiplex and coalesce messages over an unreliable stream of datagrams, for game netcode.\n\nIt's expected this will be hooked up to UDP sockets; this crate has no networking built-in.\n\n### Current Status\n\nUnder development. I'm RJ on bevy's Discord if you want to discuss.\n\nSee also: [bevy_pickleback](https://github.com/RJ/bevy_pickleback) (also very much work-in-progress)\n\nThere is an unencrypted handshake protocol for establishing a session between PiclebackServer and\nPicklebackClient. Most the the useful stuff happens when exchanging `Message` packets, after\nhandshaking. See the test harness for example usage.\n\n## Design Goals\n\nTo support multiplayer games where most updates are sent in an unreliable fashion, with occasional\nrequirements for reliable ordered messages, such as for in-game chat and on-join state synchronization.\n\nA regular exchange of packets in both directions is expected. In practice this means a packet will\nbe sent in each direction at least 10 times a second even if there are no explicit messages to transmit.\n\n\n## Features\n\n- [x] Coalesces multiple small messages into packets\n- [x] Transparently fragments \u0026 reassembles messages too large for one packet\n- [x] Multiple virtual channels for sending/receiving messages\n- [x] Optionally reliable channels, with configurable resending behaviour\n- [x] Sending a message gives you a handle to use for checking acks (even on unreliable channels)\n- [x] Internal pool of buffers for messages and packets, to minimise allocations\n- [x] No async: designed to integrate into your existing game loop. Call it each tick.\n- [x] Unit tests and integration / soak tests with bad-link simulator that drops, dupes, \u0026 reorders packets.\n- [x] Calculates rtt (need to verify outside of test harness)\n- [x] Calculate packet loss estimate (need to verify outside of test harness)\n- [ ] Enforce ordering on ordered channels (currently only reliability is supported)\n- [ ] Bandwidth tracking and budgeting\n- [ ] Allow configuration of channels and channel settings (1 reliable, 1 unreliable only atm)\n- [ ] Prioritising channels when selecting messages to send. Low volume reliables first?\n\n## TODO\n\n* Proxy some fns to ConnectedClient to tidy up serverside API\n* Benchmarks, including with and without pooled buffers.\n* Example using bevy and an unreliable transport mechanism.\n \n## Example\n\n```rust\nuse pickleback::prelude::*;\n\nlet config = PicklebackConfig::default();\nlet time = 0.0;\n\nlet mut server = PicklebackServer::new(time, \u0026config);\nlet mut client = PicklebackClient::new(time, \u0026config);\n// Address must be valid, but isn't used in this example:\nclient.connect(\"127.0.0.1:0\");\n\n// in lieu of sending over a network, deliver directly:\nfn transmit_packets(server: \u0026mut PicklebackServer, client: \u0026mut  PicklebackClient) {\n    // Server --\u003e Client\n    server.update(0.1);\n    server.visit_packets_to_send(|address, packet| {client.receive(packet, address); });\n    // Client --\u003e Server\n    client.update(0.1);\n    client.visit_packets_to_send(|address, packet| {server.receive(packet, address); });\n}\n\n// need to have some back-and-forth here to finish handshaking\n// this would usually happen each tick of your game event loop.\n\n// client sends ConnectionRequest\ntransmit_packets(\u0026mut server, \u0026mut client);\n// server responds with ConnectionChallenge\ntransmit_packets(\u0026mut server, \u0026mut client);\n// client responds with ConnectionChallengeResponse\ntransmit_packets(\u0026mut server, \u0026mut client);\n// server sends a keep-alive, denoting fully connected\ntransmit_packets(\u0026mut server, \u0026mut client);\n\nassert_eq!(client.state(), \u0026ClientState::Connected);\n\nlet channel: u8 = 0;\n\n// this can return an error if throttled due to backpressure and unable to send.\n// we unwrap here, since it will not fail at this point.\nlet msg_id = {\n    let mut connected_client = server.connected_clients_mut().next().unwrap();\n    connected_client.send_message(channel, b\"hello\").unwrap()\n};\n// server sends a Message packet containing a single message.\n// (when sending multiple messages, they are coalesced into as few packets as possible)\ntransmit_packets(\u0026mut server, \u0026mut client);\n\n// client will have received a message:\nlet mut received = client.drain_received_messages(channel);\nassert_eq!(received.len(), 1);\n// normally you'd use the .payload() to get a Reader, rather than .payload_to_owned()\n// which reads it into a Vec. But a vec here makes it easier to test.\nlet recv_payload = received.next().unwrap().payload_to_owned();\nassert_eq!(b\"hello\".to_vec(), recv_payload);\ndrop(received);\n\n// if the client doesn't have a message payload to send, it will still send\n// a packet here just to transmit acks\ntransmit_packets(\u0026mut server, \u0026mut client);\n\n// now the client has sent packets to the server, the server will have received an ack\nlet mut connected_client = server.connected_clients_mut().next().unwrap();\nassert_eq!(vec![msg_id], connected_client.drain_message_acks(channel).collect::\u003cVec\u003c_\u003e\u003e());\n```\n\n## Protocol Overview\n\nPackets consist of a packet type, sequence number, ack header, and a payload.\n\nThe ack header acknowledges receipt of the last N packets received from the remote endpoint.\n\nMessage Packet payloads consist of one or more messages. Messages can be any size, and large messages are\nfragmented into 1024 byte fragment-messages, and reassembled for you.\n\nWhen you call `send_message` you get a `MessageId`. Once the packet your message was delivered in is\nacked, you receive an ack for your `MessageId`. Unreliable channels also get acks (except if packets are lost).\n\nWhen sending a message larger than 1024 bytes, you get the ack after all fragments have been delivered,\nand the message was reassembled successfully.\n\nReliable channels will retransmit messages that remain unacked for a configurable duration.\n\nMessages are retransmitted, packets are not. ie as-yet unacked reliable mesages will be included in\nnew future packets until such time as they get acked.\n\n### Message Size\n\nArbitrarily limited to 1024 fragments of 1024B, so 1MB maximum messages size.\n\nRemember, as the number of fragments increases, the effects of packet loss are amplified.\n\n10 fragments at 1% packet loss = 1 - (0.99^10) = 9.6% chance of losing a fragment.\n\n\n### Provenance\n* [Gaffer articles](https://gafferongames.com/post/reliable_ordered_messages/) (building network protocol, packet acking, sequence buffer, etc)\n* [netcode.io rust code](https://github.com/jaynus/netcode.io/tree/master) (implementation of gaffer concepts)\n  \n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frj%2Fpickleback","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frj%2Fpickleback","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frj%2Fpickleback/lists"}