{"id":18798182,"url":"https://github.com/opencyphal-garage/cyraft","last_synced_at":"2025-04-13T17:12:43.054Z","repository":{"id":157079897,"uuid":"617476783","full_name":"OpenCyphal-Garage/cyraft","owner":"OpenCyphal-Garage","description":"Raft algorithm (for pycyphal)","archived":false,"fork":false,"pushed_at":"2024-08-01T15:09:00.000Z","size":1380,"stargazers_count":2,"open_issues_count":6,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-27T08:03:30.466Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenCyphal-Garage.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-22T13:28:36.000Z","updated_at":"2024-07-29T08:07:23.000Z","dependencies_parsed_at":null,"dependency_job_id":"b56bf72f-d09c-4f40-a9d5-decbee7ca27f","html_url":"https://github.com/OpenCyphal-Garage/cyraft","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCyphal-Garage%2Fcyraft","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCyphal-Garage%2Fcyraft/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCyphal-Garage%2Fcyraft/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenCyphal-Garage%2Fcyraft/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenCyphal-Garage","download_url":"https://codeload.github.com/OpenCyphal-Garage/cyraft/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248750127,"owners_count":21155687,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T22:11:12.024Z","updated_at":"2025-04-13T17:12:43.029Z","avatar_url":"https://github.com/OpenCyphal-Garage.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cyraft\n\nThe objective is to implement the Raft algorithm as an exercise, with the intention of incorporating [named topics](http://wiki.ros.org/Topics) into pycyphal.\n\nThis feature is significant because it enables Cyphal to serve as a communication layer between PX4 and ROS in the future.\n\n(See [UAVCAN as a middleware for ROS](https://forum.opencyphal.org/t/an-exploratory-study-uavcan-as-a-middleware-for-ros/872))\n\n- [Cyraft](#cyraft)\n  - [TODO](#todo)\n  - [Setup](#setup)\n  - [Diagrams](#diagrams)\n    - [demo\\_cyraft](#demo_cyraft)\n    - [DSDL datatypes](#dsdl-datatypes)\n  - [Sources](#sources)\n\n\n## TODO\n\n- [x] Finish study pycyphal application layer\n- [ ] `demo_cyraft.py`\n    - [x] Add instructions on how to interact with request_vote_rpc using `yakut`\n    - [x] Vscode debug setup\n    - [x] RaftNode unit tests\n      - [x] _unittest_raft_node_init\n      - [x] _unittest_raft_node_term_timeout\n      - [x] _unittest_raft_node_election_timeout\n      - [x] _unittest_raft_node_election_timeout_heartbeat\n      - [x] _unittest_raft_node_request_vote_rpc\n      - [x] _unittest_raft_node_append_entries_rpc\n    - [ ] tests\n      - [x] `leader_election.py`\n        - [ ] some warning need to be fixed here:\n          - [ ] _unittest_raft_node_term_timeout\n          - [ ] _unittest_raft_node_start_election\n      - [x] `log_replication.py`\n        - [x] `_unittest_raft_log_replication`\n        - [x] `_unittest_raft_leader_changes`\n      - [ ] `leader_commit` needs to integrated/tested\n    - [ ] Test using orchestration so there's 3 nodes running simultanously\n      - [ ] use yakut to send AppendEntries requests\n    - [ ] [Add name resolution service](https://github.com/OpenCyphal-Garage/cyraft/issues/3)\n    - [ ] [Monitor the network for online nodes](https://github.com/OpenCyphal-Garage/cyraft/issues/4)\n  - [ ] `.env-variables` and `my_env.sh` should be combined?\n  - [ ] Implement Github CI\n-  [x] Refactor code into `cyraft`\n\nQuestions:\n\n- `cyraft/node.py`:\n  - [x] how to close properly? (fixed when running a single test, however still happens when running multiple)\n- `tests/leader_election.py`:\n  - [x] difference of 1 term between node is not enough to determine who gets elected first (-\u003e tested in raft_leader_election, test stage 8/9)\n  - [x] Some strange error prinouts in test stage 10/11\n\nCode improvements:\n- [x] currently not using network to send requests, fix this\n  - https://pycyphal.readthedocs.io/en/stable/api/pycyphal.application.html#rpc-service-clients-and-servers\n  - in `_send_heartbeat`: there should be some timeout if node doesn't respond\n- [x] election and term timeouts can be done more cleanly\n- [ ] move internal variables into RaftState (for example, only leader should have a term it is able to update)\n  - Maybe not necessary? It's pretty clean as currently implemented I think.\n\n## Setup\n\n- Clone repo\n\n    ```bash\n    git clone git@github.com:OpenCyphal-Garage/cyraft.git\n    ```\n\n- Virtual environment\n\n    ```bash\n    cd ~/cyraft\n    python3 -m venv .venv\n    source .venv/bin/activate\n    ```\n \n- Install requirements (pycyphal)\n\n    ```bash\n    cd ~/cyraft\n    pip3 install -r requirements.txt\n    ```\n\n-   ```bash\n    git submodule update --init --recursive\n    ```\n\n-   ```bash\n    export CYPHAL_PATH=\"$HOME/cyraft/demo/custom_data_types:$HOME/cyraft/demo/public_regulated_data_types\"\n    ```\n\n- Set up all necessary environment variables:\n\n   ```bash\n   cd ~/cyraft\n   source my_env.sh\n    ```\n\n- Run the tests\n\n    ```bash\n    cd ~/cyraft\n    pytest tests/\n    ```\n\n    ![pytest-result](images/pytest.png)\n\n- Run the demo (**This does not work yet**)\n\n    ```bash\n    python3 demo/demo_cyraft.py\n    ```\n\n    \u003e **_NOTE:_**  Sometimes this can give an error if it's using old datatypes, try to remove ~/.pycyphal and recompile DSDL datatypes (running previous command will do this automatically)\n    \u003e   ```bash\n    \u003e   rm -rf ~/.pycyphal\n    \u003e   ```\n\n## Diagrams\n\n### demo_cyraft\n\n```mermaid\n---\ntitle: cyraft node X\n---\nflowchart TB\n    subgraph 1X:org.opencyphal.pycyphal.raft.node\n        direction TB\n        subgraph heartbeat_publisher\n            direction TB\n            heartbeat_publisher_1[/uavcan.node.Heartbeat.1.0\\]\n        end\n        heartbeat_publisher --\u003e uavcan.node.heartbeat\n        subgraph request_vote_rpc\n            direction TB\n            request_vote_1{{sirius_cyber_corp.RequestVoteRPC}}\n        end\n        10X:sirius_cyber_corp.RequestVote.Request --\u003e request_vote_rpc\n        request_vote_rpc --\u003e 10X:sirius_cyber_corp.RequestVote.Response\n        subgraph append_entries_rpc\n            direction TB\n            append_entries_1{{sirius_cyber_corp.AppendEntriesRPC}}\n        end\n        11X:sirius_cyber_corp.AppendEntriesRPC.Request --\u003e append_entries_rpc\n        append_entries_rpc --\u003e 11X:sirius_cyber_corp.AppendEntriesRPC.Response\n    end\n```\n\n### DSDL datatypes\n\n```mermaid\n---\ntitle: RequestVote\n---\nclassDiagram\n    class RequestVote_Request{\n        -uint64 term\n        -uint64 last_log_index\n        -uint64 last_log_term\n    }\n\n    class RequestVote_Response{\n        -uint64 term\n        -bool vote_granted\n    }\n```\n\n```mermaid\n---\ntitle: AppendEntries\n---\nclassDiagram\n    class AppendEntries_Request{\n        -uint64 term\n        -uint64 prev_log_index\n        -uint64 prev_log_term\n        -uint64 leaderCommit\n        -LogEntry.1.0[\u003c=1] log_entry\n    }\n\n    class AppendEntries_Response{\n        -uint64 term\n        -bool success\n    }\n```\n\n## Sources\n\n[Raft paper](https://raft.github.io/raft.pdf)\n\n[lynix94/pyraft](https://github.com/lynix94/pyraft)\n\n[zhebrak/raftos](https://github.com/zhebrak/raftos)\n\n[dronecan/libuavcan](https://github.com/dronecan/libuavcan/tree/main/libuavcan/include/uavcan/protocol/dynamic_node_id_server/distributed)\n\n[An exploratory study: UAVCAN as a middleware for ROS](https://forum.opencyphal.org/t/an-exploratory-study-uavcan-as-a-middleware-for-ros/872)\n\n[Allocators explanation in OpenCyphal/public_regulated_data_types](https://github.com/OpenCyphal/public_regulated_data_types/blob/master/uavcan/pnp/8165.NodeIDAllocationData.2.0.dsdl)\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencyphal-garage%2Fcyraft","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopencyphal-garage%2Fcyraft","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopencyphal-garage%2Fcyraft/lists"}