{"id":26326173,"url":"https://github.com/open-web3-stack/polkadot-ecosystem-tests","last_synced_at":"2025-09-11T23:15:14.754Z","repository":{"id":242798752,"uuid":"810568530","full_name":"open-web3-stack/polkadot-ecosystem-tests","owner":"open-web3-stack","description":"Polkadot Ecosystem Tests powered by Chopsticks","archived":false,"fork":false,"pushed_at":"2025-03-13T06:19:14.000Z","size":4987,"stargazers_count":13,"open_issues_count":25,"forks_count":11,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-13T07:24:59.781Z","etag":null,"topics":["polkadot"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/open-web3-stack.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-05T00:28:48.000Z","updated_at":"2025-03-13T06:19:18.000Z","dependencies_parsed_at":"2024-07-07T00:28:48.301Z","dependency_job_id":"c9ec6bc5-efe1-4fdf-abef-1f037795d217","html_url":"https://github.com/open-web3-stack/polkadot-ecosystem-tests","commit_stats":null,"previous_names":["open-web3-stack/e2e-test","open-web3-stack/polkadot-ecosystem-tests"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-web3-stack%2Fpolkadot-ecosystem-tests","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-web3-stack%2Fpolkadot-ecosystem-tests/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-web3-stack%2Fpolkadot-ecosystem-tests/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/open-web3-stack%2Fpolkadot-ecosystem-tests/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/open-web3-stack","download_url":"https://codeload.github.com/open-web3-stack/polkadot-ecosystem-tests/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243783424,"owners_count":20347404,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["polkadot"],"created_at":"2025-03-15T19:35:30.830Z","updated_at":"2025-09-11T23:15:14.727Z","avatar_url":"https://github.com/open-web3-stack.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Polkadot Ecosystem Tests\n\nAutomated testing suite for the Polkadot ecosystem powered by [Chopsticks](http://github.com/AcalaNetwork/chopsticks).\n\n## Quick Start\n\n```bash\n# Install dependencies\nyarn install\n\n# Run all tests\nyarn test\n\n# Run tests for specific chain\nyarn test \u003cchain\u003e\n\n# Run specific test\nyarn test -t \u003ctest-name\u003e #\u003ctest-name\u003e is what was passed to `vitest.test()`, usually inside a `describe()` suite\n\n# Run with Vitest UI\nyarn test:ui\n\n# Update snapshots\nyarn test -u\n```\n\n## For Network Operators\n\n### Automated Test Runs\n- Tests run every 6 hours automatically\n- Failed tests are retried after 5 minutes\n- Persistent failures trigger notifications\n- Subscribe to [notification issues](https://github.com/open-web3-stack/polkadot-ecosystem-tests/issues?q=is%3Aissue+is%3Aopen+label%3Anotifications) for updates\n\n### Manual Test Triggers\nUse the [bot trigger issue](https://github.com/open-web3-stack/polkadot-ecosystem-tests/issues/45) to run tests on-demand via GitHub Actions\n\n## For Test Writers\n\n### Test Naming Convention\n\nFor XCM tests between two networks, use the naming convention\n\n```\n\u003cchain-a\u003e.\u003cchain-b\u003e.test.ts\n```\n\nFor end-to-end tests on a single network, use\n\n```\n\u003cchain-name\u003e.\u003ctest-suite-name\u003e.e2e.test.ts\n```\n\n#### Reason\n\nThis repository has automated test failure reporting enabled.\nEach network has its own issue, to which the failing CI/CD pipeline will write a comment with the offending job.\n\nThis naming convention is necessary so the script that does all this can correctly identify the issue it must write to.\n\n### Environment Configuration\nCreate `.env` file with:\n```env\n# Required settings\nDB_PATH=./db.sqlite         # Cache database location\nRUNTIME_LOG_LEVEL=3         # Log level (1=error to 5=trace)\nLOG_LEVEL=info             # General logging (error/warn/info/debug/trace)\n\n# Optional overrides\n\u003cNETWORK\u003e_BLOCK_NUMBER=123  # Custom block number\n\u003cNETWORK\u003e_WASM=/path/to/wasm # Custom runtime\n\u003cNETWORK\u003e_ENDPOINT=wss://... # Custom endpoint\n```\n\n### Project Structure\n- `packages/shared/src/xcm`: Common XCM test suites\n- `package/shared/src/*.ts`: Common utilities for E2E tests.\n- `packages/kusama/src`: Kusama network tests\n- `packages/polkadot/src`: Polkadot network tests\n\n### About end-to-end tests\n\nThis repository contains E2E tests for the Polkadot/Kusama networks.\n\nThese include:\n- E2E suite for proxy accounts:\n  - proxy account creation, removal\n  - pure proxy account creation, removal\n  - execution of proxy calls\n  - test delay in proxy actions, as well as announcement/removal, and executing of announced action\n  - proxy call filtering works both positively and negatively; in particular, for every proxy type in Polkadot/Kusama relay and system parachains, it is checked that:\n      - a proxy of a given type can always execute calls which that proxy type is allowed to execute\n      - a proxy of a given type can never execute calls that its proxy type disallowws it from running\n        - see the section below for more\n  - E2E suite for multisig accounts and operations\n    - multisig creation, approval and execution\n    - multisig cancellation, and deposit refunding\n    - diverse failure modes tested (wrong timepoints, malformed execution/approval calls)\n- E2E suite for vesting\n  - normal (signed) and forced (root) vested transfers\n  - forced (root) vesting schedule removal\n  - merger of vesting schedules\n- E2E suite for nomination pools:\n  - Creating a pool, joining it, bonding extra\n  - Setting a pool's state, metadata, roles and commission data\n  - Scenario checking an account can be at most in one pool\n- E2E suite for governance infrastructure - referenda, preimages, and conviction voting. It includes\n  - Creating a referendum for a treasury proposal, voting on it (with conviction/split/abstain)\n  - Cancelling and killing referenda with XCM root-originated calls\n  - Noting and unnoting preimages\n- E2E suite for staking infrastructure:\n  - bonding funds, declaring intent to nominate/validate, unbonding, chilling, forcibly unstaking as `Root`\n    - includes a test to fast unstaking\n  - changing global staking configs - minimum nom./val. bonds, val. commissions, nom./val. counts, etc.\n  - more complex scenarios:\n    - forcefully updating a validator's commission after an increase to the global parameters' commission\n    - chilling other participants in the system, only when the conditions for doing so are met\n- E2E suite for the task scheduling mechanism provided by the `scheduler` pallet\n  - schedule tasks, named or otherwise\n  - cancel scheduled tasks\n- E2E test suite to the people chains in both networks. This suite contains scenarios such as\n  - Adding, modifying, and removing identities\n  - Requesting judgement requests on registrars, and providing it\n  - Adding registrars to the people chain by sending, from the relay chain, an XCM call with root origin\n  - Adding, modifying, and removing subidentities for an account\n\nThe intent behind these end-to-end tests is to cover the basic behavior of relay chains' and system\nparachains' runtimes.\n\nInitial coverage can be limited to critical path scenarios composed of common extrinsics\nfrom each of a runtime's pallets, and from there test more complex interactions.\n\nNote that since block execution throughput in `chopsticks` on a local development machine is limited\nto roughly `1-10` blocks/second, not all scenarios are testable in practice e.g. referenda\nconfirmation, or the unbonding of staked funds.\nConsider placing such tests elsewhere, or using different tools (e.g. XCM emulator).\n\n#### Proxy call filtering checker\n\nThe proxy E2E test suite contains checks to proxy types' allowed and disallowed calls - for many chains.\nBecause these tests are extensive and hard to manually verify (the test code itself and the snapshots), there exists a\ncoverage checking script (`scripts/check-proxy-coverage.ts`)\nIt searches for allowed/forbidden call coverage for a chain's proxy types.\n\nRun it with `yarn check-proxy-coverage` to see which proxy types need test coverage.\n\n### Test Guidelines\n- Write network-agnostic tests where possible\n- Handle minor chain state changes gracefully\n- Use `.redact()` for volatile values\n  - Pass `{ number: n }` to `.redact()` to explicitly redact all but the `n` most significant digits\n  - Pass `{ removeKeys: new RegExp(s) }` to remove keys from an object that are unwanted when e.g.\n    using `toMatchObject/toMatchSnapshot`. `s` can contain several fields e.g.\n    `\"alarm|index|submitted\"`. Check [this page](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/RegExp) for how to use `RegExp`.\n- Leverage snapshots for easier maintenance\n- Follow naming convention: `\u003cchain1\u003e.\u003cchain2\u003e.test.ts` or `\u003cchain1\u003e.test.ts`\n\n### Adding New Chains\n1. Add chain configuration in `packages/networks/src/chains/`\n2. Update chain index in `packages/networks/src/chains/index.ts`\n3. Create notification issue\n4. Update `.github/workflows/notifications.json`\n\n### Adding new E2E tests\n1. Create a file in `packages/shared/src/` with the E2E tests, and their required utilities\n  - This assumes that the E2E test will run on Polkadot/Kusama: its code being shared makes it\n    reusable on both chains\n2. Using the shared utilities created in the previous step, create Polkadot/Kusama tests in\n  `packages/polkadot/src`/`packages/kusama/src`, respectively.\n3. Run the newly created tests so their snapshots can be created in `packages/\u003cnetwork\u003e/src/__snapshots__`\n  - Inspect the snapshots, and make corrections to tests as necessary - or upstream, if the test\n    has revealed an issue with e.g. `polkadot-sdk`\n4. Create a PR with the new tests.\n\n### Writing Tests\n\n#### Test Structure\n\nTests are organized using Vitest and follow this general structure:\n\n```typescript\nimport { coretimePolkadot, polkadot } from '@e2e-test/networks/chains'\nimport { setupNetworks } from '@e2e-test/shared'\nimport { sendTransaction } from '@acala-network/chopsticks-testing'\nimport { check, checkEvents, checkSystemEvents } from '../helpers/index.js'\n\ndescribe('chain.otherChain', () =\u003e {\n  // Create network instance before tests\n  const [polkadotClient, coretimeClient] = await setupNetworks(polkadot, coretimePolkadot)\n\n  it('should transfer tokens', async () =\u003e {\n    // Test implementation\n    const tx = await sendTransaction(polkadotClient.api.tx.xcmPallet.teleportAssets( /* args */ ))\n\n    // Assertions using snapshots\n    await checkEvents(result).toMatchSnapshot('teleportAssets events');\n  })\n})\n```\n\n#### Key Testing Patterns\n\n1. **Network Setup**\n```typescript\n// Create chain clients\nconst [polkadotClient, coretimeClient] = await setupNetworks(polkadot, coretimePolkadot)\n```\n\n2. **Snapshot Testing**\n```typescript\n// Redact volatile values\nawait check(result).redact().toMatchSnapshot();\n\n// Redact specific digits\ncheck(result.redact({ number: 1 })).toMatchSnapshot();\n\n// Remove specific keys\ncheck(result.redact({\n  removeKeys: /(timestamp|blockHash)/\n})).toMatchSnapshot();\n```\n\n3. **Executing extrinsics with a given origin**\nUse `scheduleCallWithOrigin` on chains whose runtime includes `pallet-scheduler` to be able to execute\npermission-restricted extrinsics with the appropriate origin e.g. `Root`-gated global parameter controls\nfor staking and nomination pools\n\n4. **Send XCM `Transact`s to execute extrinsics with given origin in parachain**\nIn parachains where `pallet-scheduler` is not available, but whose relay chain has it, use `createXcmTransactSend` \nalong with `scheduleCallWithOrigin` to prepare and schedule sending of an XCM to perform the technique in 3. in the desired parachain.\n\n    4.1. Take care to adjust the parameters in accordance with the destination parachain, in particular\n         `refTime/proofSize`\n\nRead [here](https://github.com/AcalaNetwork/chopsticks?tab=readme-ov-file#testing-with-acala-networkchopsticks-testing) for more about `@acala-network/chopsticks-testing`\n\n\n#### Best Practices\n\n1. **Network-Agnostic Testing**\n   - Write shared test suites in `packages/shared/src/`\n   - Implement chain-specific tests in respective packages\n   - Use parameterized tests for multi-chain scenarios\n\n2. **Snapshot Testing**\n   - Prefer snapshots over manual assertions\n   - Redact volatile values to prevent flaky tests\n   - Review snapshot changes carefully\n   - Keep snapshots focused and readable\n   - Update snapshots when behavior changes intentionally\n\n3. **Error Handling**\n   - Test both success and failure cases\n   - Verify error messages and types\n   - Handle chain-specific error scenarios\n\n### Regenerate Snapshots\n\nIt is recommended to regenerate snapshots when renaming or removing tests. This can be done by deleting `__snapshots__` folders and running `yarn test -u`.\n\n### Debugging Tips\n- Use `{ only: true }` to isolate tests\n- Add logging to shared test suites\n- Insert `await chain.pause()` for state inspection\n- Connect via Polkadot.js Apps to paused chains\n  - Check the logs of the terminal running the `.pause`d test for the address and port\n- Try to reproduce unexpected test result in a standalone Chopsticks instance\n- Carefully review snapshot changes\n\n### Block Number Management\n```bash\n# Update KNOWN_GOOD_BLOCK_NUMBERS.env to latest\nyarn update-known-good\n\n# Update .env to latest (CI always uses KNOWN_GOOD_BLOCK_NUMBERS.env)\nyarn update-env\n```\n\n## For Maintainers\n\n### Bot Commands\n- `/bot update` - Update snapshots\n- `/bot merge` - Approve and enable auto-merge\n- `/bot cancel-merge` - Disable auto-merge\n\nAuthorized users are defined in `.github/command-runner/command-runner-config.json`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-web3-stack%2Fpolkadot-ecosystem-tests","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fopen-web3-stack%2Fpolkadot-ecosystem-tests","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fopen-web3-stack%2Fpolkadot-ecosystem-tests/lists"}