{"id":29040684,"url":"https://github.com/pushchain/push-snode","last_synced_at":"2025-06-26T14:36:38.138Z","repository":{"id":267925519,"uuid":"516271283","full_name":"pushchain/push-snode","owner":"pushchain","description":"Push Storage Node is a part of Push's Proof of Stake (PoS) chain and is responsible for storing transactions along with the payload data in a sharded fashion.","archived":false,"fork":false,"pushed_at":"2025-03-23T16:09:34.000Z","size":2135,"stargazers_count":0,"open_issues_count":25,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-23T13:54:06.956Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pushchain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-07-21T07:34:01.000Z","updated_at":"2025-03-23T16:09:40.000Z","dependencies_parsed_at":"2025-06-25T11:15:23.308Z","dependency_job_id":null,"html_url":"https://github.com/pushchain/push-snode","commit_stats":null,"previous_names":["push-protocol/push-snode","pushchain/push-snode"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/pushchain/push-snode","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pushchain%2Fpush-snode","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pushchain%2Fpush-snode/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pushchain%2Fpush-snode/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pushchain%2Fpush-snode/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pushchain","download_url":"https://codeload.github.com/pushchain/push-snode/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pushchain%2Fpush-snode/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262085900,"owners_count":23256540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-26T14:36:35.620Z","updated_at":"2025-06-26T14:36:38.114Z","avatar_url":"https://github.com/pushchain.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n    \u003ca href=\"https://push.org/#gh-light-mode-only\"\u003e\n    \u003cimg width='20%' height='10%' src=\"https://res.cloudinary.com/drdjegqln/image/upload/v1686227557/Push-Logo-Standard-Dark_xap7z5.png\"\u003e\n    \u003c/a\u003e\n    \u003ca href=\"https://push.org/#gh-dark-mode-only\"\u003e\n    \u003cimg width='20%' height='10%' src=\"https://res.cloudinary.com/drdjegqln/image/upload/v1686227558/Push-Logo-Standard-White_dlvapc.png\"\u003e\n    \u003c/a\u003e\n\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ci align=\"center\"\u003ePush protocol is evolving to Push Chain, a shared-state L1 designed to deliver universal app experiences (Any Chain. Any User. Any App).🚀\u003c/i\u003e\n\u003c/p\u003e\n\n\u003ch4 align=\"center\"\u003e\n\n  \u003ca href=\"https://discord.com/invite/pushprotocol\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/discord-7289da.svg?style=flat-square\" alt=\"discord\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://twitter.com/pushprotocol\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/twitter-18a1d6.svg?style=flat-square\" alt=\"twitter\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"https://www.youtube.com/@pushprotocol\"\u003e\n    \u003cimg src=\"https://img.shields.io/badge/youtube-d95652.svg?style=flat-square\u0026\" alt=\"youtube\"\u003e\n  \u003c/a\u003e\n\u003c/h4\u003e\n\n# Push Storage Node\n\nPush Storage Node is a part of Push's Proof of Stake (PoS) chain and is responsible for storing transactions along with the payload data in a sharded fashion.\n\n## Table of Contents\n\n- [Overview](#overview)\n- [Architecture](#architecture)\n- [Installation](#installation)\n- [Contributing](#contributing)\n- [License](#license)\n\n## Overview\n\nAfter transactions are validated by `Validator Nodes` of the PoS chain and a new block is produced, it is picked up by `Storage Nodes` for storage and indexing.\nPush Storage Nodes ensure reliable and efficient storage of transaction data by distributing the data across multiple nodes (sharding). This allows for enhanced data redundancy, fault tolerance, scalability and also ensures running a storage node is cost-effective since only a fraction of the data is stored on each node.\n\n## Architecture\n\n### Staking Requirement\n\nTo operate a storage node, participants must stake a certain amount of tokens. This staking process serves as a security deposit, ensuring that storage nodes act in the network's best interest.\n\n### Interaction in Network\n\nIn the network, once blocks are processed by validator nodes, they are picked up by storage nodes for further handling. The following diagram illustrates the interaction between the network and the storage nodes:\n\n![Network Interaction](assets/nodeInteraction.jpg)\n\n### Sharding\n\nSharding is a key component of the Push Storage Node architecture. Each node is responsible for a specific list of shards, ensuring a distributed and balanced load across the network. This approach enhances fault tolerance and scalability.\n\n- **Shard Responsibility:** Each node handles a set of shards defined by the `storage.sol` smart contract, storing the associated transaction data.\n- **Replication Factor:** To ensure data redundancy and reliability, each shard is replicated across multiple nodes. This replication factor can be adjusted based on the network requirements and is managed by the `storage.sol` smart contract.\n- **Resharding:** As more nodes join the network, resharding is performed to redistribute the shards and maintain balance. This ensures that the system can scale efficiently with the addition of new nodes.\n\n![Sharding Diagram](assets/sharding.jpg)\n\n### Indexation of Data\n\nIndexing transaction data is crucial for quick retrieval and efficient querying. The table below represents a proposed structure for transaction indexation:\n\n| wallet        | Tx Hash                              | Block Hash                           | Category | Tx Data                  | Timestamp |\n| ------------- | ------------------------------------ | ------------------------------------ | -------- | ------------------------ | --------- |\n| eip155:1:0xAA | b0249fbb-a03d-4292-9599-042c6993958e | 2608d687-fe55-4fe9-9fa5-1f782dcebb34 | email    | protobuf_serialized_data | epoch     |\n\n\u003e **Note:** The above table example is a simplified representation of the transaction indexation structure. The actual implementation may include additional fields based on the requirements of the network.\n\n## Installation\n\n⚠️ **Warning: Work In Progress** ⚠️\n\nThis project is currently a work in progress. Please be aware that things might break, and the installation process might change as we improve and dockerize it completely for public running of the node. Proceed with caution and check back frequently for updates.\n\n### Prerequisites\n\n- [Node.js](https://nodejs.org/) (\u003e= 18)\n- [Docker](https://www.docker.com/)\n- [Yarn](https://yarnpkg.com/)\n\n### Setup `do.sh` Script\n\nThe `do.sh` script is included inside the **`zips`** folder. It provides shortcuts for running various commands, including publishing a default test key and executing Hardhat with arguments. Ensure you review the code before executing any commands.\n\n- **Setting up `do.sh`**\n\n  - Place `do.sh` in a directory accessible by your environment (e.g., your home directory).\n  - Grant execute privileges to the script:\n    ```bash\n    chmod +x do.sh\n    ```\n\n- **Running `do.sh`**\n  There are multiple ways to execute the `do.sh` script:\n  - **Full Path Execution:**\n    Navigate to the project directory:\n    ```bash\n    cd /path/to/push-storage-node-project-dir /home/user/do.sh command1 command2 command3\n    ```\n  - **Add `do.sh` to Your Path:**\n    Follow the instructions in [this Apple discussion](https://discussions.apple.com/thread/254226896?sortBy=best) to add `do.sh` to your system path. Then, navigate to the project directory:\n    ```bash\n    cd /path/to/push-storage-node-project-dir\n    ./do.sh command1 command2 command3\n    ```\n  - **Create an Alias for `do.sh` (Recommended):**\n    Add an alias to your shell configuration:\n    ```bash\n    # Open .zshrc file\n    nano $HOME/.zshrc\n    # Add this line to the file\n    alias do='/Users/your-username/Documents/projects/do.sh'\n    # Save and close the file\n    ```\n    Restart your shell to apply changes. Now, you can use `do` to run commands:\n    ```bash\n    cd /path/to/push-storage-node-project-dir\n    do command1 command2 command3\n    ```\n\n### Running the Node\n\n1.  Clone the repository:\n\n    ```bash\n    git clone https://github.com/push-protocol/push-snode.git\n    cd push-snode\n    ```\n\n2.  Install dependencies:\n\n    ```bash\n    yarn install\n    ```\n\n3.  Configure docker directories:\n    To set up the storage nodes, you'll need to configure specific directories for each node. This setup ensures that each node runs independently with its own environment and key files.\n\n    - **Download and Unpack Docker Directory:**\n      Get the `docker-dir-for-snodes.zip` file from the `zips` folder and extract it into your project's root directory. After extraction, you'll find a `/docker` directory containing subdirectories for each node: `/docker/01, /docker/02`.\n      Each node directory (e.g., docker/01, docker/02) contains the necessary configuration files and scripts to run the node.\n\n    - **Key Files within Each Node Directory:**\n      This file contains environment-specific properties, such as database credentials, node identifiers, and other configuration settings that the node requires to operate.\n\n4.  Start the docker container:\n\n    ```bash\n    docker-compose up\n    ```\n\n    \u003e **Note:** It is expected that after this command you would have the following containers running: **mysql, postgres, redis**\n\n\u003c!-- 5.  MySQL Database Setup:\n    For the nodes to function correctly, you need to set up two separate MySQL databases, one for each node. These databases will store the data related to each storage node.\n\n    - Access the MySQL command-line interface by running the following command in your terminal:\n\n      ```bash\n      mysql -u root -p PASSWORD=pass\n      ```\n\n    - Once you're in the MySQL CLI, create each of the databases by running the following commands:\n\n      ```bash\n      CREATE DATABASE snode1 CHARACTER SET utf8 COLLATE utf8_general_ci;\n      CREATE DATABASE snode2 CHARACTER SET utf8 COLLATE utf8_general_ci;\n      ``` --\u003e\n\n5.  Postgres Database Setup:\n    For the nodes to function correctly, you need to set up two separate Postgres databases, one for each node. These databases will store the data related to each storage node.\n\n    - Access the pSql command-line interface by running the following command in your terminal:\n\n      ```bash\n      psql -U postgres -d postgres\n      ```\n\n    - Once you're in the pSql CLI, create each of the databases by running the following commands:\n\n      ```bash\n      create database snode1;\n      create database snode2;\n      ```\n\n6.  Run the nodes in separate terminals:\n\n    ```bash\n    # Run Storage Node 1\n    do debug.s1\n\n    # Run Storage Node 2\n    do debug.s2\n    ```\n\n## Contributing\n\nWe welcome contributions from the community! To contribute, please follow these steps:\n\n1. Fork the repository.\n2. Create a new branch (`git checkout -b feature/your-feature-name`).\n3. Make your changes and commit them (`git commit -m 'Add some feature'`).\n4. Push to the branch (`git push origin feature/your-feature-name`).\n5. Open a pull request.\n\nPlease ensure your code adheres to our coding standards and includes appropriate tests.\n\n## Licenses\n\nAll crates of this repository are licensed under either of\n\n- Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n- MIT license ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpushchain%2Fpush-snode","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpushchain%2Fpush-snode","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpushchain%2Fpush-snode/lists"}