{"id":15102697,"url":"https://github.com/artilleryio/artillery-engine-playwright","last_synced_at":"2025-09-27T00:31:28.834Z","repository":{"id":37009913,"uuid":"413512728","full_name":"artilleryio/artillery-engine-playwright","owner":"artilleryio","description":"CODE MOVED TO MAIN MONOREPO. 🌐 Load test with real web browsers! Powered by Artillery + Playwright","archived":true,"fork":false,"pushed_at":"2023-06-06T21:12:51.000Z","size":295,"stargazers_count":127,"open_issues_count":2,"forks_count":23,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-09-21T07:33:51.840Z","etag":null,"topics":["load-testing","performance-testing","playwright","testing"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/artilleryio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-04T17:03:41.000Z","updated_at":"2024-08-17T20:47:17.000Z","dependencies_parsed_at":"2022-07-11T23:33:43.459Z","dependency_job_id":null,"html_url":"https://github.com/artilleryio/artillery-engine-playwright","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/artilleryio%2Fartillery-engine-playwright","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/artilleryio%2Fartillery-engine-playwright/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/artilleryio%2Fartillery-engine-playwright/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/artilleryio%2Fartillery-engine-playwright/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/artilleryio","download_url":"https://codeload.github.com/artilleryio/artillery-engine-playwright/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":219871828,"owners_count":16554457,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["load-testing","performance-testing","playwright","testing"],"created_at":"2024-09-25T19:04:46.864Z","updated_at":"2025-09-27T00:31:28.473Z","avatar_url":"https://github.com/artilleryio.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# artillery-engine-playwright\n\n# ℹ️ The code has been moved to https://github.com/artilleryio/artillery/tree/main/packages/artillery-engine-playwright\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./header.png\" alt=\"Full browser load testing with Artillery + Playwright\" width=\"1012\"\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  Questions, comments, feedback? ➡️\u0026nbsp;\u0026nbsp;\u003ca href=\"https://github.com/artilleryio/artillery/discussions\"\u003eArtillery Discussion Board\u003c/a\u003e\n\u003c/p\u003e\n\n----\n\nEver wished you could run load tests with *real browsers*? Well, now you can.\n\nThis Artillery engine lets you combine Playwright with Artillery to be able to launch *a whole lot of real browsers* to do *full browser load testing*.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"https://www.artillery.io/img/blog/artillery-playwright-load-testing-browsers.gif\" alt=\"demo\" width=\"500\"\u003e\n\u003c/p\u003e\n\n\n## At a glance\n\n* 🤖\u0026nbsp;\u0026nbsp;\u0026nbsp;Run load tests with real (headless) Chrome instances\n* 🛰\u0026nbsp;\u0026nbsp;\u0026nbsp;Run synthetic checks in CICD with the same Artillery + Playwright scripts\n* 📊\u0026nbsp;\u0026nbsp;\u0026nbsp;See most important front-end metrics ([Largest Contentful Paint (LCP)](https://web.dev/lcp/), [First Contentful Paint (FCP)](https://web.dev/fcp/) etc) and how they are affected by high load\n* ♻️\u0026nbsp;\u0026nbsp;\u0026nbsp; Use Playwright for load testing (full access to [`page` API](https://playwright.dev/docs/api/class-page))\n* 🏎\u0026nbsp;\u0026nbsp;\u0026nbsp;Create new load testing scripts 10x faster with [`playwright codegen`](https://playwright.dev/docs/codegen-intro)\n* 🌐\u0026nbsp;\u0026nbsp;\u0026nbsp;Launch thousands of browsers, with **zero** infrastructure setup with [Artillery Pro](https://artillery.io/pro)\n\n✨ *Perfect for testing complex web apps* ✨\n\nRead the official launch blog post here: [Launching 10,000 browsers for fun and profit](https://www.artillery.io/blog/load-testing-with-real-browsers)\n\n## Why load test with browsers?\n\nLoad testing complex web apps can be time consuming, cumbersome, and brittle compared to load testing pure APIs and backend services. The main reason is that testing web apps requires a different level of abstraction: whereas APIs work at **endpoint** level, when testing web apps a **page** is a much more useful abstraction.\n\nSummarized in the table below:\n\n|    | APIs \u0026 microservices      | Web apps |\n --- | ----------- | ----------- |\n**Abstraction level**    | HTTP endpoint      | Whole page       |\n**Surface area**   | Small, a handful of endpoints        | Large, calls many APIs. Different APIs may be called depending on in-page actions by the user\n**Formal spec** | Usually available (e.g. as an OpenAPI spec) | No formal specs for APIs used and their dependencies. You have to spend time in Dev Tools to track down all API calls\n**In-page JS** | Ignored. Calls made by in-page JS have to be accounted for manually and emulated | Runs as expected, e.g. making calls to more HTTP endpoints |\n\nAll of those factors combined make load testing web apps with traditional approaches very frustrating and time consuming. 😞\n\n## Usage ⌨️\n\n### Installation\n\nInstall Artillery and this engine:\n\n```sh\nnpm install -g artillery artillery-engine-playwright\n```\n\n(See [Use in Docker/CI](#use-in-dockerci) if running tests in Docker/CI)\n\n### Running a test\n\nCreate an Artillery script:\n\n`hello-world.yml`:\n\n```yaml\nconfig:\n  target: https://www.artillery.io\n  # Enable the Playwright engine:\n  engines:\n    playwright: {}\n  processor: \"./flows.js\"\nscenarios:\n  - engine: playwright\n    flowFunction: \"helloFlow\"\n    flow: []\n```\n\nUse a Playwright script to describe virtual user scenario:\n\n(Note: this script was generated with [`playwright codegen`](https://playwright.dev/docs/cli/#generate-code). `page` is an instance of [Playwright page](https://playwright.dev/docs/api/class-page/).)\n\n`flows.js`:\n\n```js\nmodule.exports = { helloFlow };\n\nasync function helloFlow(page) {\n  //\n  // The code below is just a standard Playwright script:\n  //\n  // Go to https://artillery.io/\n  await page.goto('https://artillery.io/');\n  // Click text=Pricing\n  await page.click('text=Pricing');\n  // assert.equal(page.url(), 'https://artillery.io/pro/');\n  // Click text=Sign up\n  await page.click('text=Sign up');\n}\n```\n\nRun it:\n\n```sh\nartillery run hello-world.yml\n```\n\nArtillery runs Playwright-based scenarios, and provides user-centric metrics that measure [perceived load speed](https://web.dev/user-centric-performance-metrics/#types-of-metrics) such as LCP and FCP:\n\n```\n--------------------------------\nSummary report @ 11:24:53(+0100)\n--------------------------------\n\nvusers.created_by_name.Dev account signup: .................. 1\nvusers.created.total: ....................................... 1\nvusers.completed: ........................................... 1\nvusers.session_length:\n  min: ...................................................... 5911.7\n  max: ...................................................... 5911.7\n  median: ................................................... 5944.6\n  p95: ...................................................... 5944.6\n  p99: ...................................................... 5944.6\nbrowser.page.domcontentloaded: .............................. 2\nbrowser.page.domcontentloaded.https://artillery.io/: ........ 1\nbrowser.page.domcontentloaded.https://artillery.io/pro/: .... 1\nbrowser.page.FCP.https://artillery.io/:\n  min: ...................................................... 1521.1\n  max: ...................................................... 1521.1\n  median: ................................................... 1525.7\n  p95: ...................................................... 1525.7\n  p99: ...................................................... 1525.7\nbrowser.page.dominteractive:\n  min: ...................................................... 162\n  max: ...................................................... 1525\n  median: ................................................... 162.4\n  p95: ...................................................... 162.4\n  p99: ...................................................... 162.4\nbrowser.page.dominteractive.https://artillery.io/:\n  min: ...................................................... 1525\n  max: ...................................................... 1525\n  median: ................................................... 1525.7\n  p95: ...................................................... 1525.7\n  p99: ...................................................... 1525.7\nbrowser.page.LCP.https://artillery.io/:\n  min: ...................................................... 1521.1\n  max: ...................................................... 1521.1\n  median: ................................................... 1525.7\n  p95: ...................................................... 1525.7\n  p99: ...................................................... 1525.7\nbrowser.page.dominteractive.https://artillery.io/pro/:\n  min: ...................................................... 162\n  max: ...................................................... 162\n  median: ................................................... 162.4\n  p95: ...................................................... 162.4\n  p99: ...................................................... 162.4\nbrowser.page.FCP.https://artillery.io/pro/:\n  min: ...................................................... 205.3\n  max: ...................................................... 205.3\n  median: ................................................... 206.5\n  p95: ...................................................... 206.5\n  p99: ...................................................... 206.5\nbrowser.page.LCP.https://artillery.io/pro/:\n  min: ...................................................... 205.3\n  max: ...................................................... 205.3\n  median: ................................................... 206.5\n  p95: ...................................................... 206.5\n  p99: ...................................................... 206.5\n```\n\n## Configuration\n\nThe underlying Playwright instance may be configured through `config.engines.playwright`.\n\nYou can pass the following options in:\n\n- `launchOptions` - an object containing arguments to [`browserType.launch()`](https://playwright.dev/docs/api/class-browsertype#browser-type-launch) method\n- `contextOptions` - an object containing arguments to [`browser.newContext()`](https://playwright.dev/docs/api/class-browser#browser-new-context) method\n- `defaultNavigationTimeout` and `defaultTimeout` - a number in seconds. These values are shorthands for [`browserContext.setDefaultNavigationTimeout()`](https://playwright.dev/docs/api/class-browsercontext#browser-context-set-default-navigation-timeout) and [`browserContext.setDefaultTimeout`](https://playwright.dev/docs/api/class-browsercontext#browser-context-set-default-timeout)\n\n### Example 1: turn off headless mode\n\nYou can turn off the default headless mode to see the browser window for local debugging by setting the [`headless`](https://playwright.dev/docs/api/class-browsertype#browser-type-launch-option-headless) option.\n\n```yaml\nconfig:\n  engines:\n    playwright:\n      launchOptions:\n        headless: false\n```\n\n### Example 2: set extra HTTP headers\n\nThis example sets the [`extraHTTPHeaders`](https://playwright.dev/docs/api/class-browser#browser-new-context-option-extra-http-headers) option for the browser context that is created by the engine.\n\n```yaml\nconfig:\n  engines:\n    playwright:\n      contextOptions:\n        extraHTTPHeaders:\n          x-my-header: my-value\n```\n\n### Aggregate metrics by scenario name\n\nBy default metrics are aggregated separately for each unique URL. When load testing the same endpoint with different/randomized query params, it can be hepful to group metrics by a common name. \n\nTo enable the option pass `aggregateByName: true` to the playwright engine and give a name to your scenarios:\n```\nconfig:\n  target: https://artillery.io\n  engines:\n    playwright: { aggregateByName: true } \n  processor: \"./flows.js\"\nscenarios:\n  - name: blog\n    engine: playwright\n    flowFunction: \"helloFlow\"\n    flow: []\n```\n`flows.js`\n```\nmodule.exports = { helloFlow };\n\nfunction helloFlow(page) {\n  await page.goto(`https://artillery.io/blog/${getRandomSlug()}`);\n}\n```\n\nThis serves a similar purpose to the `useOnlyRequestNames` option from the [metrics-by-endpoint](https://github.com/artilleryio/artillery-plugin-metrics-by-endpoint) artillery plugin.\n\n## Flow function API\n\nBy default, only the `page` argument (see Playwright's [`page` API](https://playwright.dev/docs/api/class-page/)) is required for functions that implement Playwright scenarios, e.g.:\n\n```js\nmodule.exports = { helloFlow };\n\nasync function helloFlow(page) {\n  // Go to https://artillery.io/\n  await page.goto('https://artillery.io/');\n}\n```\n\nThe functions also have access to virtual user context and events arguments, which can be used to access scenario variables for different virtual users, or to [track custom metrics](https://artillery.io/docs/guides/guides/extending.html#Tracking-custom-metrics).\n\n```js\nmodule.exports = { helloFlow };\n\nasync function helloFlow(page, vuContext, events) {\n  // Increment custom counter:\n  events.emit('counter', 'user.page_loads', 1);\n  // Go to https://artillery.io/\n  await page.goto('https://artillery.io/');\n}\n```\n\n\n\n## More examples\n\nSee [Artillery + Playwright examples](https://github.com/artilleryio/artillery/tree/master/examples/browser-load-testing-playwright) in the main `artillery` repo.\n\n## Use in Docker/CI\n\nUse the [`Dockerfile`](./Dockerfile) which bundles Chrome, Playwright and Artillery to run your tests in CI.\n\n**Note:** To keep the Docker image small, browsers other than Chromium are removed (the saving is ~500MB)\n\n## Performance (Does it scale?)\n\nWhen you run load tests with browsers your **main bottleneck** is going to be memory. Memory usage is what will limit how many Chrome instances can **run in parallel** in each VM/container. (Remember that every virtual user (VUs) will run its own copy of Chrome - just like in the real world.)\n\nTwo factors will determine how many VUs will be able to run in parallel:\n\n1. Memory usage of the pages that the VU navigates to and uses. Lightweight pages will mean each VU uses less memory while it's running.\n1. How long each VU runs for - the longer each session, the more VUs will be running at the same time, the higher memory usage will be.\n\nThe engine tracks a `browser.memory_used_mb` metric which shows the distribution of memory usage across all pages in each scenario (in megabytes). The value is read from [`window.performance.memory.usedJSHeapSize`](https://developer.mozilla.org/en-US/docs/Web/API/Performance/memory) API. The metric is recorded for every [`load` event](https://developer.mozilla.org/en-US/docs/Web/API/Window/load_event).\n\nUltimately, there is no formula to determine how many VUs can be supported on a given amount of memory unfortunately as it will depend on the application and VU scenario. The rule of thumb is to scale out horizontally and run with as much memory as possible and track VU session length and memory usage of Chrome to be able to tweak the number of containers/VMs running your tests.\n\n## Questions, comments, feedback?\n\n➡️\u0026nbsp;\u0026nbsp;\u0026nbsp;Let us know via \u003ca href=\"https://github.com/artilleryio/artillery/discussions\"\u003eArtillery Discussion board\u003c/a\u003e\n\n\n----\n\n\n## License 📃\n\nMPL 2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fartilleryio%2Fartillery-engine-playwright","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fartilleryio%2Fartillery-engine-playwright","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fartilleryio%2Fartillery-engine-playwright/lists"}