{"id":29192858,"url":"https://github.com/rugpullindex/indexed-sparse-merkle-tree","last_synced_at":"2025-07-02T02:01:52.004Z","repository":{"id":45369233,"uuid":"430131637","full_name":"attestate/indexed-sparse-merkle-tree","owner":"attestate","description":"A dapptools-ready and gas-optimized implementation of a sparse merkle tree in Solidity. ","archived":false,"fork":false,"pushed_at":"2022-11-17T20:32:10.000Z","size":205,"stargazers_count":96,"open_issues_count":10,"forks_count":8,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-25T16:02:33.874Z","etag":null,"topics":["ethereum","gas-optimization","merkle-tree","merkletree","solidity","sparse-merkle-tree"],"latest_commit_sha":null,"homepage":"","language":"Solidity","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/attestate.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-11-20T14:57:23.000Z","updated_at":"2025-02-17T22:05:46.000Z","dependencies_parsed_at":"2023-01-22T13:16:05.421Z","dependency_job_id":null,"html_url":"https://github.com/attestate/indexed-sparse-merkle-tree","commit_stats":null,"previous_names":["rugpullindex/indexed-sparse-merkle-tree"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/attestate/indexed-sparse-merkle-tree","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attestate%2Findexed-sparse-merkle-tree","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attestate%2Findexed-sparse-merkle-tree/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attestate%2Findexed-sparse-merkle-tree/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attestate%2Findexed-sparse-merkle-tree/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/attestate","download_url":"https://codeload.github.com/attestate/indexed-sparse-merkle-tree/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/attestate%2Findexed-sparse-merkle-tree/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263061400,"owners_count":23407605,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ethereum","gas-optimization","merkle-tree","merkletree","solidity","sparse-merkle-tree"],"created_at":"2025-07-02T02:01:34.333Z","updated_at":"2025-07-02T02:01:51.656Z","avatar_url":"https://github.com/attestate.png","language":"Solidity","funding_links":[],"categories":["Data Structures","Libraries"],"sub_categories":["Merkle Trees"],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"/assets/logo.png\" /\u003e\n\u003c/p\u003e\n\n[![lint: prettier-solidity](https://github.com/attestate/indexed-sparse-merkle-tree/actions/workflows/lint.yml/badge.svg)](https://github.com/attestate/indexed-sparse-merkle-tree/actions/workflows/lint.yml) [![unit tests](https://github.com/attestate/indexed-sparse-merkle-tree/actions/workflows/main.yml/badge.svg)](https://github.com/attestate/indexed-sparse-merkle-tree/actions/workflows/main.yml)\n\n### A dapptools-ready and gas-optimized implementation of a sparse merkle tree in Solidity. \n#### [Installation](readme.md/#Installation) | [Usage](readme.md/#Usage) | [Contributing](readme.md/#Contributing) | [Gas-usage Benchmark Results](.gas-benchmark)\n\n## Why use it?\n\n- This implementation is **[gas\n  optimized](https://ethresear.ch/t/optimizing-sparse-merkle-trees/3751)**\n  according to a ethresear.ch post from Vitalik Buterin and others.\n- It's [dapptools](https://github.com/dapphub/dapptools)-compatible and so all\n  tests are written in Solidity: **Maintenance is improved**.\n\n## Installation\n\n### Prerequisites\n\n- You need to have [dapptools](https://github.com/dapphub/dapptools) installed.\n\n### Including as a dapptools dependency\n\n```bash\ndapp install attestate/indexed-sparse-merkle-tree\n```\n\nOnce you've done that, you can use the `StateTree.sol` contract class\n`StateTree` to initialize the merkle tree.\n\n## Usage\n\nHere's an example of how `StateTree.sol` could be used. Here,\n`StateTree.empty()` computes the default empty sparse merkle tree. That root is\nthen stored in a `bytes32` storage variable when `Dapp` is initialized.\n\n```sol\npragma solidity ^0.8.6;\nimport \"indexed-sparse-merkle-tree/StateTree.sol\";\n\ncontract Dapp {\n  bytes32 public root;\n\n  constructor() {\n    root = StateTree.empty();\n  }\n}\n```\n\n## How Do Sparse Merkle Trees Work?\n\nSparse Merkle Trees (short: SMT) are regular merkle trees (or sometimes called\n\"hash trees\"). For brevity, we're not going to go into details about how merkle\ntrees work, but only what the \"sparse\" attribute means.\n\nGenerally speaking, a merkle tree has, as a base layer, a number of leaves that\nare being hashed. But the core assumption of a merkle tree is that most or all\nleaves on the base layer have data in them. Here's a classical example of a\nbinary hash tree.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"/assets/hashtree.svg\" /\u003e\n\u003c/p\u003e\n\nNow for sparse merkle tree, the default assumption is that all leaves at the\ntree's birth are empty. And this empty value is usually represented by some\nkind of value. In programming languages terms like `void`, `null`, `nil` are\noften used to represent emptiness. However, since we want to hash all leaf\nvalues using a hash function like e.g. sha3, we'll have to somehow represent\nemptiness nominally. Hence, when calling `StateTree.empty()`, the tree's\ninitializing function, we generate a list of leaves all empty and represented\nby zeros:\n\n```\nleaves = [0, 0, ...,0];\n```\n\nAnd we then hash those leaves exactly the same way we'd hash a regular merkle\ntree.\n\n### So why use a **sparse** merkle tree in the first place?\n\nA merkle tree is great for validating the integrity of an existing file. E.g.\nin Bittorrent, when someone posts a magnet link to a movie, then knowing the\nmovie file's hash is useful as it helps to verify the integrity of the movie\ncurrently downloaded.\n\nBut e.g. in the case of storing data structures on-chain as is done with\ndecentralized apps, a sparse merkle tree that is initially empty but can be\nupdated to contain more and more values, is kinda neat.\n\nE.g. after we initialized our tree with `StateTree.empty()`, we can then write\nvalues into any of the leaves by calling `StateTree.write()` to receive a new\nroot. An example:\n\n```sol\nbytes32[] memory proofs = new bytes32[](0);\nbytes32 OLD_LEAF_HASH = keccak256(\n  abi.encode(\n    0x0000000000000000000000000000000000000000000000000000000000000000\n  )\n);\nbytes32 NEW_LEAF_HASH = keccak256(\n  abi.encode(\n    0x0000000000000000000000000000000000000000000000000000000000000001\n  )\n);\nuint8 bits = 0;\nuint256 index = 0;\n\nbytes32 ROOT0 = StateTree.empty();\nbytes32 ROOT1 = StateTree.write(\n  proofs,\n  bits,\n  index,\n  NEW_LEAF_HASH,\n  OLD_LEAF_HASH,\n  ROOT0\n);\n```\n\nSee, in the above example, we proposed overwriting the old value of the tree\n`OLD_LEAF_HASH` with a new value `NEW_LEAF_HASH`. We said that in the tree\nitself we want to write the leaf at `index = 0`. There's also two variables not\nbeing used yet: `proofs` and `bits`.\n\nNormally, for each insertion to have the `write` function succeed, we have to\nprovide a valid old merkle root hash. This is because internally, `write` first\nchecks with `validate` if by reinserting `OLD_LEAF_HASH` we get the same old\nroot as we claimed with `ROOT0`.\n\nOnly if that's the case, we're allowed to proceed to actually attempt\ncomputation of the new root `ROOT1`.\n\nIn smart contract programming, originally (sparse) merkle trees were used to\nvalidate integrity. But with the rise of the rollup infrastructures, it's\nbecome known that SMTs might also allow to build linearly scalable dapps by\nsaving gas costs when storing words in the EVM's state.\n\n### Why sparse merkle trees could safe you gas\n\nStoring a non-zero word of 32 bytes for the first time in EVM storage currently\ncosts 20k gas. However, including many 32bytes leaves into an SMT might cost\nless than 20k gas per 32bytes. A user or operator would still have to pay for\nthe tree's root inclusion into the chain, e.g. by storing it as a storage\nvariable - but even for large body of data, cost would decrease. Generally\nspeaking, most cost would occur when having to compute the new root.\n\n### Using `write` with `uint8 bits`\n\nA special property of this sparse merkle tree is that it is gas-optimized\naccording to a post from\n[ethresear.ch](https://ethresear.ch/t/optimizing-sparse-merkle-trees/3751). In\ngeneral, a sparse merkle tree of binary structure has a mandatory proof size of\nits depth. In simpler terms, this means that for an SMT with depth eight, for\nany type of recomputation eight other nodes on various levels would have to be\nnecessary. Actually, there's a really great article by Vitalik called\n\"[Merkling in\nEthereum](https://blog.ethereum.org/2015/11/15/merkling-in-ethereum/) that may\nhelp to understand this property.\n\nBut in any case, at e.g. a depth of 256 and having 2^256 leaves, requiring 256\nproofs of 32 bytes ends up being a lot of data; that doesn't sound gas\nefficient at all. And that were Vitalik's post on \"[Optimizing sparse Merkle\ntrees](https://ethresear.ch/t/optimizing-sparse-merkle-trees/3751)\" comes in\nhandy. Because, if we assume that most of our tree will mostly remain\nrelatively empty, then we can precompute the leaves that just represent zero\nvalues on all sorts of levels.\n\nFor that, e.g. imagine a sparse merkle tree with just a single non-zero leaf at\nthe left hand side. If we looked at it, we'd immediately see that all other\nleaf nodes are zero hashes. We'd also understand that even for the most top\nright node, closest to the root node, it could be precomputed too and would\nsimply be the accumulation of all zero node hashes on the last seven levels.\n\nIndeed, the `StateTree` has all these values available precomputed in the\n`get(uint256 level)` function. And that's how we can safe gas costs, by\nreducing the number of proofs required to update the tree. Going back to our\nexample earlier:\n\n```sol\nbytes32[] memory proofs = new bytes32[](0);\nbytes32 OLD_LEAF_HASH = keccak256(\n  abi.encode(\n    0x0000000000000000000000000000000000000000000000000000000000000000\n  )\n);\nbytes32 NEW_LEAF_HASH = keccak256(\n  abi.encode(\n    0x0000000000000000000000000000000000000000000000000000000000000001\n  )\n);\nuint8 bits = 0;\nuint256 index = 0;\n\nbytes32 ROOT0 = StateTree.empty();\nbytes32 ROOT1 = StateTree.write(\n  proofs,\n  bits,\n  index,\n  NEW_LEAF_HASH,\n  OLD_LEAF_HASH,\n  ROOT0\n);\n```\n\nWe can see that `proofs` and `bits` is actually empty, as all zero-value hashes\ncan be used from within the actual implementation. So not a single proof has to\nbe put into calldata.\n\nIf, however, we were to now enter a second value at another index into the tree\nwe'd have to start getting creative. See, `bits` indicates at which level we\nhave to send in a custom proof and can't use a pre-computed zero proof.\n\n```sol\n// continued from code snippet above...\n\nuint256 proofIndex = 0;\nuint8 bytePos = (bufLength - 1) - (proofIndex / 8);\nuint8 bits2 = bytePos + 1 \u003c\u003c (proofIndex % 8);\n\nbytes32[] memory proofs2 = new bytes32[](1);\nproofs2[0] = NEW_LEAF_HASH;\n\nuint256 index2 = 1;\nbytes32 ROOT2 = StateTree.write(\n  proofs2,\n  bits2,\n  index2,\n  NEW_LEAF_HASH,\n  OLD_LEAF_HASH,\n  ROOT1\n);\n```\n\nSo in this case, we're hence making the `write` function understand that at\n`proof2[0]`, there's a non-zero proof for the computation. We've done so by\ncomputing a bit map on a `uint8 bits2` value. This looks like some rather\nintimidating math, but what it really does is it flips the first zero value of\nan integer represented in binary to a one. E.g. `0000` to `1000`.\n\nFor the user's convenience however, we're shipping this bitmap function in the\ncode too. Just call `bitmap(uint256 index)`.\n\n## Contributing\n\nYou can download this repository and test the implementation using the\nfollowing command:\n\n```bash\ndapp test -vv\n```\n\n## LICENSE\n\nSee LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frugpullindex%2Findexed-sparse-merkle-tree","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frugpullindex%2Findexed-sparse-merkle-tree","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frugpullindex%2Findexed-sparse-merkle-tree/lists"}