{"id":15923929,"url":"https://github.com/ppopth/big-calldata-shadow","last_synced_at":"2025-04-03T12:45:34.795Z","repository":{"id":122591437,"uuid":"596348630","full_name":"ppopth/big-calldata-shadow","owner":"ppopth","description":null,"archived":false,"fork":false,"pushed_at":"2023-03-19T08:48:22.000Z","size":56,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-09T02:15:10.892Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ppopth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-02T01:32:48.000Z","updated_at":"2023-03-19T08:48:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"cea422cb-129d-4aef-ab49-92984c8910c8","html_url":"https://github.com/ppopth/big-calldata-shadow","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppopth%2Fbig-calldata-shadow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppopth%2Fbig-calldata-shadow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppopth%2Fbig-calldata-shadow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ppopth%2Fbig-calldata-shadow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ppopth","download_url":"https://codeload.github.com/ppopth/big-calldata-shadow/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247005397,"owners_count":20868019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T21:03:08.789Z","updated_at":"2025-04-03T12:45:34.772Z","avatar_url":"https://github.com/ppopth.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Simulate Ethereum network with big calldata\n\nWe want to run the simulation see the network effect of producing a large block. Currently with the Bellatrix fork, we can do that by sending\na transaction with big calldata.\n\nThis simulation relies heavily on [ethereum-shadow](https://github.com/ppopth/ethereum-shadow). It's also helpful to read more about it before\nrunning the simulation.\n\n## Installation\n\nPlease follow https://github.com/ppopth/ethereum-shadow#install-dependencies to install the required dependencies. Then follow the following steps.\n\n```bash\n# Install Solidity\nsudo add-apt-repository ppa:ethereum/ethereum\nsudo apt-get update\nsudo apt-get install solc\n\n# Install Go\ncurl -OL https://go.dev/dl/go1.20.linux-amd64.tar.gz\nsudo tar -C /usr/local -xzf go1.20.linux-amd64.tar.gz\n# If /usr/local/go/bin is not already in your PATH, run the following command\necho 'export PATH=\"${PATH}:/usr/local/go/bin\"' \u003e\u003e ~/.bashrc \u0026\u0026 source ~/.bashrc\n\ngit clone https://github.com/ppopth/big-calldata-shadow.git\ncd big-calldata-shadow\ngit submodule update --init --depth 1\n\n# Build the customized geth\ncd go-ethereum\nmake geth\ncd ..\n\n# Install node modules\nnpm install\n```\n\n## Run the simulation\n\n```bash\n./run.sh\n```\nBy default, the number of nodes will be 10, the number of validators will be 100, and the size of the calldata will be 2MB.\nYou can change them by setting the environment variables.\n```bash\n# 102400 is 100KB\nLENGTH=102400 NODE_COUNT=20 VALIDATOR_COUNT=150 ./run.sh\n```\n\n## Network topology\n\nCurrently we set each node to have 20Mbps for downloading and another 20Mbps for uploading. Each pair of nodes has the latency of 100ms\n(which is the average latency of the nodes in discv5 network, see https://notes.ethereum.org/@pop/discv5-network-measurement).\n\nIn the future, we have a plan to assign each node the physical location so that the latencies among nodes will be more realistic.\n\n## Customized go-ethereum\n\nYou can note that we have go-ethereum as a submodule and we compile it from source. That is because we want to patch go-ethereum in order to\n1. Increase all the limits that are necessary to send a large transaction.\n2. Disable transaction broadcasting because our transaction is large and it will eat bandwidth in the Execution Layer instead of just the Consensus Layer.\n\n## What happens in the network\n\nIn order to send a transaction with big calldata, we need to have a contract that accepts a big-calldata transaction first. The Solidity code of the\ncontract looks as follows.\n\n```solidity\ncontract BytesContract {\n    uint public size;\n    function update(bytes calldata b) external {\n        size = b.length;\n    }\n}\n```\n\nAs mentioned earlier, we disable transaction broadcasting in go-ethereum, so, when we need to send a transaction to the chain, it will be on the chain\nonly when the node to which we send the transaction is the proposer. So, if we send a transaction to a node, we need to wait until that node becomes\na proposer.\n\nIn order to make sure the contract is deployed quickly, we have every node create and send a contract-creation transaction and include it in the block\nwhen it becomes a proposer. This results to a lot of contracts created in the block chain (I know that there is a way to create only one contract, but\nit's easy to do this for now). You can see the lines like the following when you run the simulation.\n```\nAdded contract deploy process in node1\nAdded contract deploy process in node2\nAdded contract deploy process in node3\n...\n```\n\nAfter the contract is deployed, we will send a big-calldata transaction which will call the `update` function in the contract. For the same reason, this\ntime, we also have many nodes create and send the transaction as well (but not every node because otherwise we have too many big-calldata transactions\ncreated in the network). We do it in enough nodes just to have less than 10% of probability that there is no big-calldata transaction included in the\nblock chain at all. If you run the simulation and you don't see any such transaction in the chain at all, it means that you are in that 10% and\nyou should run the simulation again. You can see the lines like the following if the node is assigned to create a transaction.\n```\nAdded bytes update process in node1\n```\n\nYou can see the transaction of which node is included in the chain by looking at the file `./data/shadow/hosts/node{id}/node{id}.node.1005.stdout`.\nIf the transaction is included in the chain, the file should look like this.\n```\n7:06:00 AM\ncontract_address 0x43f352Cca3ca64eDbD1f5fD47cedf9C63eC9aCf4\ntransaction 0x00a9139d0a7c60442448e0df6917e97b7ec8aa099403a71a036920c5cbb04945\nblock_number 49\nblock_hash 0x77907894c6d4956a4a58f43bbd5208f3477a7b7893e211f48c60f1d0a075ef23\nstatus true\n7:09:36 AM\n```\nIf the transaction is not included, the file will not have the `block_number`, `block_hash`, and the `status`.\n\n## Simulation result\n\nPlease look at https://github.com/ppopth/ethereum-shadow#simulation-result to see the relevant files you probably want to look at.\n\n## Scale to hundreds of nodes\n\nPlease look at https://github.com/ppopth/ethereum-shadow#scale-to-hundreds-of-nodes to see how to scale the simulation to a lot of nodes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppopth%2Fbig-calldata-shadow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fppopth%2Fbig-calldata-shadow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fppopth%2Fbig-calldata-shadow/lists"}