{"id":47612192,"url":"https://github.com/rostmanrk/playwright-orchestrator","last_synced_at":"2026-04-22T11:03:17.718Z","repository":{"id":272637304,"uuid":"916741222","full_name":"rostmanrk/playwright-orchestrator","owner":"rostmanrk","description":"Library to orchestrate your playwright tests","archived":false,"fork":false,"pushed_at":"2026-03-22T19:23:44.000Z","size":880,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-23T03:58:59.196Z","etag":null,"topics":["e2e","e2e-testing","playwright","test-orchestration"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/@playwright-orchestrator/core?activeTab=readme","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rostmanrk.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-01-14T17:17:07.000Z","updated_at":"2026-03-22T19:22:49.000Z","dependencies_parsed_at":"2025-03-26T21:34:31.230Z","dependency_job_id":null,"html_url":"https://github.com/rostmanrk/playwright-orchestrator","commit_stats":null,"previous_names":["rostmanrk/playwright-orchestrator"],"tags_count":12,"template":false,"template_full_name":null,"purl":"pkg:github/rostmanrk/playwright-orchestrator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rostmanrk%2Fplaywright-orchestrator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rostmanrk%2Fplaywright-orchestrator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rostmanrk%2Fplaywright-orchestrator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rostmanrk%2Fplaywright-orchestrator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rostmanrk","download_url":"https://codeload.github.com/rostmanrk/playwright-orchestrator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rostmanrk%2Fplaywright-orchestrator/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31291699,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["e2e","e2e-testing","playwright","test-orchestration"],"created_at":"2026-04-01T20:34:09.462Z","updated_at":"2026-04-18T14:13:15.180Z","avatar_url":"https://github.com/rostmanrk.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Playwright Orchestrator\n\nA distributed, queue-based test runner for Playwright — replaces static sharding with dynamic worker scheduling.\n\n[![npm version](https://img.shields.io/npm/v/@playwright-orchestrator/core)](https://www.npmjs.com/package/@playwright-orchestrator/core)\n[![License](https://img.shields.io/npm/l/@playwright-orchestrator/core)](LICENSE.md)\n[![CI](https://github.com/rostmanrk/playwright-orchestrator/actions/workflows/pr.yml/badge.svg)](https://github.com/rostmanrk/playwright-orchestrator/actions/workflows/pr.yml)\n[![Node.js](https://img.shields.io/node/v/@playwright-orchestrator/core)](https://nodejs.org)\n\n## The Problem\n\nPlaywright's `--shard` assigns tests to shards before execution starts. The assignment is fixed: if one shard draws the slow tests, it runs long while the others finish and sit idle. You can't rebalance mid-run, you can't retry just the failures without re-running the whole shard, and there's no history to learn from.\n\nPlaywright Orchestrator replaces pre-assignment with a shared queue. Workers pull tests at runtime — fast workers pick up more work automatically. Slow tests, flaky tests, and machine variance stop blocking your CI.\n\n## Quick Start\n\nEach `run` command is a worker. Start as many as you need — on one machine or distributed across a fleet.\n\nLinks to full CI workflow examples: [MongoDB](.github/workflows/mongo.yml) · [DynamoDB](.github/workflows/dynamo-db.yml) · [PostgreSQL](.github/workflows/pg.yml)\n\n**1. Initialize storage** (one-time, requires write table permissions):\n\n```bash\nnpx playwright-orchestrator init pg --connection-string \"postgres://username:password@localhost:5432/postgres\"\n```\n\n**2. Create a run** (once per CI run, outputs a `runId`):\n\n```bash\nnpx playwright-orchestrator create pg --connection-string \"postgres://username:password@localhost:5432/postgres\" --workers 2\n\u003e 019464f7-1488-75d1-b5c0-5e7d3dde9195\n```\n\n**3. Start workers** (run in parallel — each pulls tests from the queue independently):\n\n```bash\nnpx playwright-orchestrator run pg --connection-string \"postgres://username:password@localhost:5432/postgres\" --run-id 019464f7-1488-75d1-b5c0-5e7d3dde9195\n```\n\n**4. Re-run failed tests** — repeat step 3 using the same `runId` (rerun any shard). The tool automatically picks up only failed tests from the previous run.\n\n**5. Merge reports** using [Playwright's merge-reports CLI](https://playwright.dev/docs/test-sharding#merge-reports-cli):\n\n```bash\nnpx playwright merge-reports ./blob-reports --reporter html\n```\n\n## How It Works\n\n```\nShared Queue (Redis / PostgreSQL / MongoDB / ...)\n       ↓              ↓              ↓\n  [Worker 1]     [Worker 2]     [Worker N]\n       ↓              ↓              ↓\n  Playwright     Playwright     Playwright\n```\n\nEach worker runs the same `playwright-orchestrator run` command and loops independently:\n\n1. Pull the next test (or batch) from the shared queue\n2. Execute with Playwright, emit a blob report\n3. Report the result — duration and pass/fail update the scheduling model\n4. Repeat until the queue is empty\n\nWorkers don't coordinate directly. Fast workers naturally pick up more tests. Slow machines or flaky tests don't block the others.\n\n### EMA Scheduling\n\n**Tests are ordered as follows:**\n\n1. If there are no previous test runs, use the default timeout (runtime timeouts are ignored, see [this PR](https://github.com/rostmanrk/playwright-orchestrator/pull/34)).\n2. Take the [EMA (Exponential Moving Average)](https://en.wikipedia.org/wiki/Moving_average#Exponential_moving_average) of the test duration. The window size can be changed in [`create`](#create) command, default value is 10. **Smaller value strips previous history.**\n3. If there was a failed test in the chosen window, it is more likely to fail again. Therefore, the formula is adjusted as follows:\n\n    - Calculate the EMA of the test duration.\n    - Adjust the EMA by adding a factor based on the percentage of failed tests in the window.\n    - The final formula is: `EMA + EMA * (% of failed tests in window)`\n\n    **Example:**\n\n    - If the EMA of the test duration is 2 minutes and 4 of 10 tests in the window failed, the adjusted duration would be:\n      `2 + 2 * 0.4 = 2.8 minutes`\n\n**Serial tests duration is a sum of all included tests.**\n\n### Test identifiers\n\nIn order to keep history of tests, they need to be identified. There are multiple possible cases:\n\n1. Default id: `{file} \u003e {title of the test}`.\n2. Serial test id: `{file} \u003e {title of the parent group}`.\n3. Serial test defined at the file level: `{file}`.\n4. Custom Id: `{custom id}`.\n5. `[{project}] ` added to beginning if `--grouping project`\n\nIn case some of these attributes changed, test history would be recreated. To preserve test history between changes, you can add a **static** attribute. Adding an id to an existing test would recreate the history as well.\n\n**✅ Examples**\n\n```\nimport { id } from '@playwright-orchestrator/core/annotations';\ntest('test', { annotation: id('#custom_id') }, () =\u003e {\n    // test code\n}\n```\n\n```\nimport { id } from '@playwright-orchestrator/core/annotations';\ntest.describe('serial tests', { annotation: id('#custom_id') }, () =\u003e {\n    // test code\n}\n```\n\n**❌ This won't work**\n\n```\nimport { id } from '@playwright-orchestrator/core/annotations';\ntest.describe('test', () =\u003e {\n    test.info().annotations.push(id('#custom_id'));\n    // test code\n}\n```\n\n## Storage Adapters\n\n- file: Basic storage that uses a local file as storage. Created purely for testing, but still may work for someone.\n- dynamo-db: Amazon's DynamoDB adapter.\n- pg: PostgreSQL adapter.\n- mysql: MySQL adapter.\n- mongo: MongoDB adapter.\n- redis: Redis adapter.\n\nEach adapter is an optional peer dependency — install only what you need.\n\n## 📦 Installation\n\nMake sure Playwright is installed by following [Playwright's installation guide](https://playwright.dev/docs/intro#installation).\n\n```bash\nnpm install @playwright-orchestrator/core --save-dev\n\nnpm install @playwright-orchestrator/\u003cstorage_plugin_name\u003e --save-dev\n```\n\n## Why Not Native Sharding?\n\nPlaywright's `--shard` pre-assigns tests before execution. Playwright Orchestrator uses a shared queue workers pull from at runtime:\n\n![Timeline](https://github.com/rostmanrk/playwright-orchestrator/raw/main/assets/timeline.png)\n\n| Feature            | Playwright `--shard`       | Playwright Orchestrator                           |\n| ------------------ | -------------------------- | ------------------------------------------------- |\n| Execution model    | Pre-assigned static shards | Workers pull from shared queue                    |\n| Split method       | By test count              | By predicted duration (EMA)                       |\n| Dynamic balancing  | No — fixed at run start    | Yes — fast workers pick up more                   |\n| Flakiness-aware    | No                         | Yes — flaky tests scheduled first                 |\n| Rerun strategy     | Re-run entire shard        | Start a new worker for failed tests only          |\n| Persistent history | No                         | Yes — ordering improves over time                 |\n| Storage            | None                       | File, PostgreSQL, MySQL, MongoDB, DynamoDB, Redis |\n\n## Batching and Grouping\n\n### Batching\n\nBatching groups multiple tests into a single Playwright process invocation. This reduces per-test overhead and is most effective when individual tests are short and launch time dominates runtime.\n\n**`--batch-mode off` (default):** Each test runs in its own process. Safe for all workloads.\n\n**`--batch-mode time`:** Groups tests until their predicted total duration approximately reaches `--batch-target` seconds. Produces batches of roughly equal wall-clock length — best for even distribution across shards.\n\n**`--batch-mode count`:** Groups exactly `--batch-target` tests per batch. Predictable batch sizes regardless of individual test duration.\n\n### Grouping\n\n**`--grouping test` (default):** Each test is a separate scheduling unit \\* each project. Most granular control; recommended for most setups.\n\n**`--grouping project`:** Tests are grouped by Playwright project before scheduling. Uses a less optimized query. Only consider this if your workload has a specific reason to prefer project-level grouping — the default (`test`) is recommended.\n\n## ⚙️ Commands and Options\n\n### `init`\n\nSeeds data storage with necessary tables and initial configuration.\nNo additional options.\n\n### `create`\n\nCreates and configures a new test run. Outputs created run ID. Supports most of [playwright's options](https://playwright.dev/docs/test-cli#reference).\n\n| Option             | Description                                                                                                                          | Type                   | Default | Required?                            |\n| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------ | ---------------------- | ------- | ------------------------------------ |\n| `--history-window` | Count of runs history kept and window for average duration. More [here](#how-it-works)                                               | `number`               | `10`    | no                                   |\n| `--batch-mode`     | Batch grouping mode. `off` uses current single-test behaviour                                                                        | `off \\| time \\| count` | `off`   | no                                   |\n| `--batch-target`   | Batch size: seconds (time mode) or test count (count mode)                                                                           | `number`               | -       | yes when `--batch-mode` is not `off` |\n| `--grouping`       | How tests are grouped. Experiment on your workload, but per project query is less optimized, using default behaviour is recommended. | `test \\| project`      | `test`  | no                                   |\n\n### `run`\n\nStarts a test shard for the provided test run. If used with a finished run, it will only start failed tests.\n\nCommand generate blob reports into `--output` directory. To merge it use [Playwright's Merge-reports CLI](https://playwright.dev/docs/test-sharding#merge-reports-cli)\n\n| Option                   | Description                                                                                                                                                       | Type     | Default        | Required? |\n| ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------- | -------------- | --------- |\n| `--run-id`               | Run ID generated by `create` command                                                                                                                              | `string` | -              | yes       |\n| `-o, --output`           | Directory for artifacts produced by tests                                                                                                                         | `string` | `blob-reports` | no        |\n| `--shard-id`             | Identifier for this shard instance (alphanumeric, hyphens, underscores only). Used to track per-shard start/finish times in reports. Defaults to a random UUID v7 | `string` | auto           | no        |\n| `--fail-on-test-failure` | Exit with code `1` if at least one test failed                                                                                                                    | -        | -              | no        |\n\n### `create-report`\n\nGenerates report for provided test run id.\n\n| Option                   | Description                                    | Type              | Default | Required? |\n| ------------------------ | ---------------------------------------------- | ----------------- | ------- | --------- |\n| `--run-id`               | Run ID generated by `create` command           | `string`          | -       | yes       |\n| `--reporter`             | Type of reporter                               | `'json' \\| 'gha'` | `json`  | no        |\n| `--fail-on-test-failure` | Exit with code `1` if at least one test failed | -                 | -       | no        |\n\n![Example](https://github.com/rostmanrk/playwright-orchestrator/raw/main/assets/report-example.png)\n\nThe report includes a **shard summary** table showing each shard's start time, finish time, duration, and the discrepancy between the fastest and slowest shard — useful for identifying load imbalance.\n\n## ⚙️ Subcommands and Options\n\nEach command has corresponding subcommands for installed storages.\n\n**Each storage option can be parsed from env variable**. For example, `table-name-prefix` -\u003e TABLE_NAME_PREFIX.\n\n### `file`\n\nUse as a storage locally created file\n\n| Option        | Description                      | Type     | Default   | Required? |\n| ------------- | -------------------------------- | -------- | --------- | --------- |\n| `--directory` | Directory to store test run data | `string` | test-runs | no        |\n\n### `dynamo-db`\n\nUse Amazon's DynamoDB as storage. Credentials are taken from AWS profile\n\n| Option                | Description           | Type     | Default                   | Required? |\n| --------------------- | --------------------- | -------- | ------------------------- | --------- |\n| `--table-name-prefix` | Table(s) name prefix  | `string` | 'playwright-orchestrator' | no        |\n| `--ttl`               | TTL in days           | `number` | 30                        | no        |\n| `--endpoint-url`      | DynamoDB endpoint URL | `string` | -                         | no        |\n\n### `pg`\n\nUse PostgreSQL as storage.\n\n| Option                | Description          | Type     | Default                   | Required? |\n| --------------------- | -------------------- | -------- | ------------------------- | --------- |\n| `--connection-string` | Connection string    | `string` | -                         | yes       |\n| `--table-name-prefix` | Table(s) name prefix | `string` | 'playwright-orchestrator' | no        |\n| `--ssl-ca `           | SSL CA               | `string` | -                         | no        |\n| `--ssl-cert `         | SSL certificate      | `string` | -                         | no        |\n| `--ssl-key `          | SSL key              | `string` | -                         | no        |\n\n### `mysql`\n\nUse MySQL as storage.\n\n| Option                            | Description                                  | Type     | Default                   | Required? |\n| --------------------------------- | -------------------------------------------- | -------- | ------------------------- | --------- |\n| `--connection-string`             | Connection string                            | `string` | -                         | yes       |\n| `--table-name-prefix`             | Table(s) name prefix                         | `string` | 'playwright-orchestrator' | no        |\n| `--ssl-profile `                  | The SSL profile overrides other SSL options. | `string` | -                         | no        |\n| `--ssl-ca`                        | SSL CA                                       | `string` | -                         | no        |\n| `--ssl-cert`                      | SSL certificate                              | `string` | -                         | no        |\n| `--ssl-key`                       | SSL key                                      | `string` | -                         | no        |\n| `--ssl-passphrase`                | SSL passphrase                               | `string` | -                         | no        |\n| `--ssl-reject-unauthorized`       | SSL reject unauthorized                      | -        | -                         | no        |\n| `--ssl-verify-server-certificate` | SSL verify server certificate                | -        | -                         | no        |\n\n### `redis`\n\nUse Redis as storage.\n\n| Option                | Description         | Type     | Default | Required? |\n| --------------------- | ------------------- | -------- | ------- | --------- |\n| `--connection-string` | Connection string   | `string` | -       | yes       |\n| `--name-prefix`       | Records name prefix | `string` | `'pw'`  | no        |\n| `--ttl`               | TTL in days         | `number` | 30      | no        |\n\n### `mongo`\n\nUse MongoDB as storage.\n\n| Option                             | Description                           | Type     | Default                   | Required? |\n| ---------------------------------- | ------------------------------------- | -------- | ------------------------- | --------- |\n| `--connection-string`              | Connection string                     | `string` | -                         | yes       |\n| `--db`                             | Database name                         | `string` | -                         | yes       |\n| `--collection-name-prefix`         | Table(s) name prefix                  | `string` | 'playwright-orchestrator' | no        |\n| `--tls`                            | Enable TLS                            | -        | -                         | no        |\n| `--tls-ca`                         | SSL CA                                | `string` | -                         | no        |\n| `--tls-key`                        | SSL key                               | `string` | -                         | no        |\n| `--tls-key-password`               | SSL key password                      | `string` | -                         | no        |\n| `--tls-passphrase`                 | SSL passphrase                        | `string` | -                         | no        |\n| `--tls-allow-invalid-certificates` | Allow invalid certificates            | -        | -                         | no        |\n| `--tls-allow-invalid-hostnames`    | Allow invalid hostnames               | -        | -                         | no        |\n| `--tls-insecure`                   | Allow insecure                        | -        | -                         | no        |\n| `--debug`                          | Add extra fields for some collections | `string` | -                         | no        |\n\n## 🔄 Migration Guide\n\n### Upgrading to v1.4\n\n**All SQL storage adapters (`pg`, `mysql`) require re-running `init` to apply schema updates** (adds the `shards` column to the test runs table).\n\n### Upgrading to v1.3\n\n**SQL storage adapters (`pg`, `mysql`) require re-running `init` to apply schema updates.**\n\n### v1.3.3 Updates\n\n`webServer` is now fully supported.\n`repeatEach` is now properly handled: status is considered as `Passed` when any of the repeats is successful.\n\n## Adding new adapter\n\nIf you need specific adapters create an issue, I will implement once I have time. Or you are welcome to contribute.\n\n## 💻 Development\n\nMake sure podman and compose is installed. They used for tests and local development.\n\nBuild with `pnpm build` or use `pnpm watch`.\n\nSee packages.json .scripts section for more commands.\n\n## Contributing\n\nIssues and pull requests are welcome. If something doesn't work as expected, please [open an issue](https://github.com/rostmanrk/playwright-orchestrator/issues).\n\nThe best way to support this project is to star it on GitHub and share it with your colleagues or the community.\n\n## 🔮 Future plans/ideas\n\n- ⬜ Create Documentation site.\n- ❓ Restore unfinished tests in case shard terminated (Can be simply fixed by creating new run)\n\n## ⚖️ License\n\nLicensed under the Apache License 2.0. See LICENSE.md for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frostmanrk%2Fplaywright-orchestrator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frostmanrk%2Fplaywright-orchestrator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frostmanrk%2Fplaywright-orchestrator/lists"}