{"id":14958564,"url":"https://github.com/ethereum/execution-spec-tests","last_synced_at":"2026-04-21T14:04:26.291Z","repository":{"id":65516936,"uuid":"553749291","full_name":"ethereum/execution-spec-tests","owner":"ethereum","description":"A Python framework and collection of test cases to generate test vectors for Ethereum execution clients","archived":false,"fork":false,"pushed_at":"2026-03-25T17:03:18.000Z","size":9639153,"stargazers_count":202,"open_issues_count":1,"forks_count":195,"subscribers_count":15,"default_branch":"main","last_synced_at":"2026-04-01T09:54:18.655Z","etag":null,"topics":["ethereum","evm","python"],"latest_commit_sha":null,"homepage":"https://eest.ethereum.org/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethereum.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2022-10-18T17:52:59.000Z","updated_at":"2026-03-25T17:17:59.000Z","dependencies_parsed_at":"2026-01-30T15:04:57.682Z","dependency_job_id":null,"html_url":"https://github.com/ethereum/execution-spec-tests","commit_stats":{"total_commits":1168,"total_committers":36,"mean_commits":32.44444444444444,"dds":0.6626712328767124,"last_synced_commit":"6ae70a77482a46e1815c2ed6c9ab8a29250caa7e"},"previous_names":[],"tags_count":140,"template":false,"template_full_name":null,"purl":"pkg:github/ethereum/execution-spec-tests","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethereum%2Fexecution-spec-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethereum%2Fexecution-spec-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethereum%2Fexecution-spec-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethereum%2Fexecution-spec-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethereum","download_url":"https://codeload.github.com/ethereum/execution-spec-tests/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethereum%2Fexecution-spec-tests/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290741,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ethereum","evm","python"],"created_at":"2024-09-24T13:17:24.496Z","updated_at":"2026-04-01T18:01:43.262Z","avatar_url":"https://github.com/ethereum.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# ⚠️ NOTICE: All code from this repository has been migrated to ethereum/execution-specs\n\nAs such, this repository is no longer accepting pull requests as of 2025-11-01. Please direct any changes to https://github.com/ethereum/execution-specs instead.\n\n## What This Means for You\n\n- **Client Teams:** No changes. Test fixture releases will continue to be published in this repository, [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests).\n- **Test Contributors:**\n    - Please direct PRs to [ethereum/execution-specs](https://github.com/ethereum/execution-specs).\n    - Read more about the changes effecting test development in our [blog announcement](https://steel.ethereum.foundation/blog/2025-11-04_weld_final/).\n\n### Questions or Feedback About the Migration?\n\nPlease reach out to the STEEL team by writing to the `#el-testing` channel in the [Ethereum R\u0026D Discord Server](https://discord.com/invite/qGpsxSA) or contacting any STEEL team member directly.\n\n---\n\n## Execution Spec Tests\n\n[![latest version](https://img.shields.io/github/v/release/ethereum/execution-spec-tests)](https://github.com/ethereum/execution-spec-tests/releases/latest)\n![Python Versions](https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue)\n[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)\n[![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://github.com/astral-sh/uv)\n[![License](https://img.shields.io/github/license/ethereum/execution-spec-tests)](https://github.com/ethereum/execution-spec-tests/blob/main/LICENSE)\n\n## About\n\nThe full execution-spec-tests documentation can be found [here](https://eest.ethereum.org/main/).\n\n**Note:** All of [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) functionality (i.e., the spec tests, test frameworks and tooling) will move to [ethereum/execution-specs](https://github.com/ethereum/execution-specs) in Q4 2025 to streamline spec and test development. For more information please see our [blog post](https://steel.ethereum.foundation/blog/2025-09-11_weld-announcement/). For now, please continue contributing to both [ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and [ethereum/execution-specs](https://github.com/ethereum/execution-specs) as you previously have - we will reach out to you if and when need be!\n\n[ethereum/execution-spec-tests](https://github.com/ethereum/execution-spec-tests) is both a collection of test cases and a framework implemented in Python to generate tests for Ethereum execution clients.\n\nThe framework collects and executes the test cases in order to generate _test fixtures_ (JSON) which can be consumed by any execution client to verify their implementation of [ethereum/execution-specs](https://github.com/ethereum/execution-specs). The fixtures, which define state transition and block tests, are generated by the framework using one of the `t8n` command-line tools that are provided by most execution clients, see below for an overview of the supported `t8n` tools.\n\n```mermaid\n---\ntitle: Test Fixture Generation with execution-spec-tests\n---\nflowchart LR\n  style C stroke:#333,stroke-width:2px\n  style D stroke:#333,stroke-width:2px\n  style G stroke:#F9A825,stroke-width:2px\n  style H stroke:#F9A825,stroke-width:2px\n\n  subgraph \"ethereum/go-ethereum, ...\"\n    C[\u003ccode\u003eevm t8n\u003c/code\u003e\u003cbr/\u003eexternal executable]\n  end\n\n  subgraph ethereum/solidity\n    D[\u003ccode\u003esolc\u003c/code\u003e\u003cbr/\u003eexternal executable]\n  end\n\n  subgraph ethereum/EIPs\n    E(\u003ccode\u003eEIPS/EIP-*.md\u003c/code\u003e\u003cbr/\u003eSHA digest via Github API)\n  end\n\n  subgraph \"ethereum/execution-spec-tests\"\n    A(\u003ccode\u003e./tests/**/*.py\u003c/code\u003e\u003cbr/\u003ePython Test Cases)\n    B([\u003ccode\u003e$ fill ./tests/\u003c/code\u003e\u003cbr/\u003ePython Framework])\n  end\n\n  subgraph Test Fixture Consumers\n    subgraph ethereum/hive\n      G([\u003ccode\u003e$ hive ...\u003c/code\u003e\u003cbr/\u003eGo Test Framework])\n    end\n    H([Client executables])\n  end\n\n  C \u003c-.-\u003e B\n  D \u003c-.-\u003e B\n  A --\u003e B\n  E \u003c-.-\u003e |retrieve latest spec version\\ncheck tested spec version| B\n  B --\u003e|output| F(\u003ccode\u003e./fixtures/**/*.json\u003c/code\u003e\\nJSON Test Fixtures)\n  F --\u003e|input| G\n  F --\u003e|input| H\n```\n\nThe generated test fixtures can be used:\n\n1. Directly by client teams' test frameworks, and,\n2. In the integration tests executed in the [ethereum/hive](https://github.com/ethereum/hive) framework.\n\n## Transition Tool Support\n\nThe following transition tools are supported by the framework:\n\n| Client | \"t8n\" Tool | Tracing Support |\n| -------| ---------- | --------------- |\n| [ethereum/evmone](https://github.com/ethereum/evmone) | `evmone-t8n` | Yes |\n| [ethereum/execution-specs](https://github.com/ethereum/execution-specs) | `ethereum-spec-evm` | Yes |\n| [ethereum/go-ethereum](https://github.com/ethereum/go-ethereum) | [`evm t8n`](https://github.com/ethereum/go-ethereum/tree/master/cmd/evm) | Yes |\n| [ethereumjs](https://github.com/ethereumjs/ethereumjs-monorepo) | [`ethereumjs-t8ntool.sh`](https://github.com/ethereumjs/ethereumjs-monorepo/tree/master/packages/vm/test/t8n) | No |\n| [hyperledger/besu](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | [`evm t8n-server`](https://github.com/hyperledger/besu/tree/main/ethereum/evmtool) | Yes |\n| [status-im/nimbus-eth1](https://github.com/status-im/nimbus-eth1) | [`t8n`](https://github.com/status-im/nimbus-eth1/blob/master/tools/t8n/readme.md) | Yes |\n\n### Upcoming EIP Development\n\nGenerally, specific `t8n` implementations and branches must be used when developing tests for upcoming EIPs.\n\nWe use named reference tags to point to the specific version of the `t8n` implementation that needs to be used fill the tests.\n\nAll current tags, their t8n implementation and branch they point to, are listed in [.github/configs/evm.yaml](.github/configs/evm.yaml).\n\n## Getting Started\n\n### Prerequisites\n\nThe tools provided by ethereum/execution-spec-tests use `uv` ([docs.astral.sh/uv](https://docs.astral.sh/uv/)) to manage their dependencies and virtual environment. `uv` downloads Python for your target platform if one of the required versions (Python 3.11 or 3.12) is not available natively.\n\n`uv` can be installed via curl (recommended; can self-update):\n\n```console\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n```\n\nor pip (requires Python, can't self-update):\n\n```console\npip install uv\n```\n\n### Installation\n\nClone [execution-spec-tests](https://github.com/ethereum/execution-spec-tests) and install its dependencies:\n\n```console\ngit clone --depth 1 https://github.com/ethereum/execution-spec-tests\ncd execution-spec-tests\nuv python install 3.12\nuv python pin 3.12\nuv sync --all-extras\n```\n\nSee [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) in the online docs if you encounter issues.\n\n### Exploring and Filling Test Cases\n\nBy default, JSON test fixtures are generated from this repository's Python test cases using the [Ethereum Execution Layer Specification](https://github.com/ethereum/execution-specs) (EELS) reference implementation. The resulting JSON fixtures can be executed against execution clients to verify consensus. The process of generating fixtures is often referred to as \"filling\".\n\n1. Explore test cases via `--collect-only` and search for test cases that combine `PUSH0` and `DELEGATECALL` in the EVM functionality introduced in the Shanghai hard fork:\n\n    ```console\n    uv run fill --collect-only -k \"push0 and delegatecall\" tests/shanghai/\n    ```\n\n    The `fill` command is based on [`pytest`](https://docs.pytest.org/en/stable/). The above command uses the [optional pytest arguments](https://docs.pytest.org/en/stable/how-to/usage.html):\n\n    - `--collect-only` only collect test cases; don't execute them.\n    - `-k \u003cexpression\u003e` filter test cases by their test case ID based on the given expression.\n    - `tests/shanghai` the directory containing the test cases (tells `fill` to only discover test cases in this directory; default: `tests/`).\n\n    Expected console output:\n    ![Screenshot of pytest test collection console output](./docs/filling_tests/img/pytest_collect_only.png)\n\n2. Fill `state_test` fixtures for these test cases:\n\n    ```console\n    uv run fill -k \"push0 and delegatecall\" tests/shanghai/ -m state_test -v\n    ```\n\n    where:\n\n    - `-m state_test` only fills test cases marked as a `state_test` (see all available markers via `uv run fill --markers`).\n    - `-v` enables verbose output.\n\n    Expected console output:\n    ![Screenshot of fill test collection console output](./docs/filling_tests/img/pytest_run_example.png)\n\n3. Verify the generated fixtures:\n\n    a. Check the corresponding fixture file has been generated:\n\n    ```console\n    head fixtures/state_tests/shanghai/eip3855_push0/push0/push0_contract_during_call_contexts.json\n    ```\n\n    b. Open the generated HTML test using the link provided at the bottom of the console output. This is written to the output directory at:\n\n    ```console\n    ./fixtures/.meta/report_fill.html\n    ```\n\n## Usage\n\nMore information on how to obtain and consume the [released test fixtures](https://github.com/ethereum/execution-spec-tests/releases) can be found in the [documentation](https://eest.ethereum.org/main/running_tests/).\n\nFor further help with working with this codebase, see the [online documentation](https://eest.ethereum.org/main/):\n\n1. Learn [useful command-line flags](https://eest.ethereum.org/main/filling_tests/filling_tests_command_line/).\n2. [Execute tests for features under development](https://eest.ethereum.org/main/filling_tests/filling_tests_dev_fork/) via the `--from=FORK1` and `--until=FORK2` flags.\n3. _Optional:_ [Configure VS Code](https://eest.ethereum.org/main/getting_started/setup_vs_code/) to auto-format Python code and [execute tests within VS Code](https://eest.ethereum.org/main/filling_tests/filling_tests_vs_code/).\n4. Implement a new test case, see [Writing Tests](https://eest.ethereum.org/main/writing_tests/).\n\n## Coverage\n\nThe available test cases can be browsed in the [Test Case Reference doc](https://eest.ethereum.org/main/tests/).\n\n## Installation Troubleshooting\n\nIf you encounter issues during the installation process, please refer to the [Installation Troubleshooting](https://eest.ethereum.org/main/getting_started/installation_troubleshooting/) page.\n\n## Contributing\n\nContributions and feedback are welcome. Please see our [Contributing Guidelines](CONTRIBUTING.md) for detailed information on how to contribute, and the [online documentation](https://eest.ethereum.org/main/writing_tests/) for this repository's coding standards and help on implementing new tests.\n\nWe welcome earnest newcomers, no matter how small the contribution! However, we do not accept:\n\n- Contributions that only fix spelling or grammatical errors in documentation, code or elsewhere\n- Pull requests from obvious airdrop farmers\n- Drive-by or vibe code contributions without proper engagement or context\n\nPull requests should have reasonable substance or resolve an existing repository open issue.\n\nCare is required when adding PRs or issues for functionality that is live on Ethereum mainnet, please refer to the [Security Policy](SECURITY.md) for more information about reporting vulnerabilities and eligibility for the [bug bounty program](https://bounty.ethereum.org).\n\n## License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethereum%2Fexecution-spec-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethereum%2Fexecution-spec-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethereum%2Fexecution-spec-tests/lists"}