{"id":28150006,"url":"https://github.com/harsh-ps-2003/pikachu","last_synced_at":"2025-05-15T02:15:13.633Z","repository":{"id":288218925,"uuid":"920727208","full_name":"harsh-ps-2003/pikachu","owner":"harsh-ps-2003","description":"A simple DHT implementation!","archived":false,"fork":false,"pushed_at":"2025-04-16T06:18:27.000Z","size":768,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-16T07:56:25.465Z","etag":null,"topics":["chord","consistent-hashing","hashtable","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/harsh-ps-2003.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-22T17:08:40.000Z","updated_at":"2025-04-16T06:24:04.000Z","dependencies_parsed_at":"2025-04-16T12:00:33.935Z","dependency_job_id":null,"html_url":"https://github.com/harsh-ps-2003/pikachu","commit_stats":null,"previous_names":["harsh-ps-2003/pikachu"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harsh-ps-2003%2Fpikachu","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harsh-ps-2003%2Fpikachu/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harsh-ps-2003%2Fpikachu/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/harsh-ps-2003%2Fpikachu/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/harsh-ps-2003","download_url":"https://codeload.github.com/harsh-ps-2003/pikachu/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254259380,"owners_count":22040822,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chord","consistent-hashing","hashtable","rust"],"created_at":"2025-05-15T02:15:06.621Z","updated_at":"2025-05-15T02:15:13.624Z","avatar_url":"https://github.com/harsh-ps-2003.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Pikachu \n\nAn asynchronous multi-dimensional in-memory non-persistent Distributed Hash Table implementation.\n\n### Demo\n\nTo start the bootstrap node, run `cargo run start-bootstrap -p \u003cBOOTSTRAP_NODE_PORT\u003e`. If the port `-p` is omitted, a random available port might be used. Check whether the bootstrap node is working properly or not by curling the gRPC server of the bootstrap node `lsof -i :\u003cNODE_PORT\u003e`.\n\nThen join this bootstrap node to make a chord network `cargo run join -b \u003cBOOTSTRAP_NODE_PORT\u003e -p \u003cNODE_PORT\u003e`. Similar to starting, if the local port `-p` is omitted, a random one might be assigned. The nodes will automatically detect each other, and start forming the chord network.\n\nTake a look at `\u003cNODE_PORT\u003e.log` files for the logs. You can control the log verbosity using the `RUST_LOG` environment variable (e.g., `RUST_LOG=debug cargo run ...` for detailed logs, defaults to `info`).\n\nTo put the key-value pair in the DHT use `cargo run put -p \u003cNODE_PORT\u003e -k \u003cKEY\u003e -v \u003cVALUE\u003e`. This connects to the node running on `127.0.0.1:\u003cNODE_PORT\u003e`.\nSimilarly to get it back from DHT use `cargo run get -p \u003cNODE_PORT\u003e -k \u003cKEY\u003e`.\n\nJust `^C` for graceful shutdown of node, and then close the terminal instance.\n\nAnd when you only spawn 2/3 nodes, you will see a lot of failing routing table updates. Chord requires a sufficient number of nodes, spread across the ID space, to populate the finger tables effectively. With only one node, most `find_successor` calls will fail because there's no other node to point to. With two nodes, you'll have some entries, but many will still be missing. Don't worry :)\n\nAnd yes, aren't there too many logs when you use `RUST_LOG=debug` with the command!\n\n### System Design\n\n\u003e [!WARNING]\n\u003e This implementation is **not crash fault tolerant**. Node failures can lead to data loss or network instability.\n\n#### Successor/Predecessor List Size\n\nThe Chord protocol typically recommends successor/predecessor list sizes (`r`) on the order of O(log N), where N is the number of nodes. For simplicity in this local development setup, `r` is fixed to a small constant (e.g., 3), as a large number of nodes is not anticipated.\n\n#### Lookup Mechanism\n\nThis implementation uses **Recursive Lookups** for finding the node responsible for a given key.\n\n**How it Works:**\n\n1.  **Initiation:** The querying node identifies the closest preceding node to the target key ID in its finger table.\n2.  **Forwarding:** It sends the lookup request to that node.\n3.  **Processing:** The receiving node checks if *it* is the successor. If not, it finds the closest preceding node in *its* finger table and forwards the request again.\n4.  **Repetition:** This forwarding continues hop-by-hop closer to the target ID.\n5.  **Resolution:** The node ultimately responsible for the key identifies itself and returns the resultback to the node that called it.\n6.  **Return Path:** Each node in the chain then receives this response and forwards it back up the call stack to the node that originally requested it from them. The response effectively retraces the request path in reverse.\n\nTo understand in the architecture in detail, [read all the chapters](/architecture/). \n\n\u003c!-- ### Something extra...\n\nThis is a very close implementation of the infamous Chord DHT with a twist, I have made it BFT using SMPC protocol. --\u003e\n\n### Things to Further think about...\n\n* Instead of hopping the response back, we should communicate directly. Using the DHT primarily to find the responsible node's address and then communicating directly is often more efficient than having the response hop back through the entire chain.\n* Need a careful review to ensure correctness under various conditions (e.g., joining an empty vs. populated network, concurrent joins).\n* Ensuring that key transfers during joins/stabilization are atomic or at least robust against failures is crucial for data consistency. If any node along the forwarding path fails, the lookup request can be lost. Without built-in retry mechanisms at each hop or by the originator, the request fails.\n* Intermediate nodes handle both their own load and forwarded requests, potentially creating bottlenecks.\n\n### Reference\n\nThanks to the authors of these wonderful research papers for inspiring me for this personal project :\n\n* [Chord](https://pdos.csail.mit.edu/papers/chord:sigcomm01/chord_sigcomm.pdf)\n* [How to make Chord correct](https://arxiv.org/pdf/1502.06461)\n* [A Statistical Theory of Chord under Churn](https://arxiv.org/pdf/cs/0501069)\n* [Atomic Data Access in DHTs](https://groups.csail.mit.edu/tds/papers/Lynch/lncs02.pdf)\n* [Making Chord Robust to Byzantine Attacks](https://www.cs.unm.edu/~saia/papers/swarm.pdf)\n* [Towards Practical Communication in Byzantine-Resistant DHTs](https://www.cs.purdue.edu/homes/akate/publications/RobustP2P.pdf)\n* [A survey of DHT security](https://dl.acm.org/doi/pdf/10.1145/1883612.1883615)\n* [Building p2p systems with Chord, a Distributed Lookup Service](https://www.cs.princeton.edu/courses/archive/spr05/cos598E/bib/dabek-chord.pdf)\n* [Comparing Performance of DHTs under churn](https://pdos.csail.mit.edu/~strib/docs/dhtcomparison/dhtcomparison-iptps04.pdf)\n* [Design and Analysis in Structures p2p systems](https://dcatkth.github.io/thesis/sameh_thesis.pdf)\n\n### Some Improvements...\n\nSome cool things that can be further done :\n\n* Make this CFT via proper replication factor\n* Enable TLS in gRPC communication for better security\n* Make this BFT\n* Make this DHT privacy-preserving as well - [Add Query Privacy to Robust DHTs](https://arxiv.org/pdf/1107.1072)\nThat would be cool :)\n* Sybil attacks could poison the network as nodes can join without authentication\n* Adding some property tests would be cool!\n\n### Disclaimer \n\nThis project was undertaken to deepen my understanding of Decentralized P2P networks after taking the course EE698C at IIT Kanpur.\n\nNot designed to be used in production!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharsh-ps-2003%2Fpikachu","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fharsh-ps-2003%2Fpikachu","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fharsh-ps-2003%2Fpikachu/lists"}