{"id":26085232,"url":"https://github.com/semrush/purr","last_synced_at":"2025-04-12T02:01:29.900Z","repository":{"id":37167780,"uuid":"215523938","full_name":"semrush/purr","owner":"semrush","description":"PURR (PUppeteer RunneR) is a devops-friendly tool for browser testing and monitoring.","archived":false,"fork":false,"pushed_at":"2024-12-12T15:50:22.000Z","size":1650,"stargazers_count":41,"open_issues_count":6,"forks_count":9,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-25T21:47:11.662Z","etag":null,"topics":["puppeteer","purr"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/semrush.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-10-16T10:45:24.000Z","updated_at":"2025-01-12T18:45:16.000Z","dependencies_parsed_at":"2024-11-15T16:33:56.080Z","dependency_job_id":null,"html_url":"https://github.com/semrush/purr","commit_stats":null,"previous_names":[],"tags_count":58,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semrush%2Fpurr","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semrush%2Fpurr/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semrush%2Fpurr/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/semrush%2Fpurr/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/semrush","download_url":"https://codeload.github.com/semrush/purr/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248505862,"owners_count":21115354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["puppeteer","purr"],"created_at":"2025-03-09T05:57:49.333Z","updated_at":"2025-04-12T02:01:29.871Z","avatar_url":"https://github.com/semrush.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PURR\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n\n- [PURR](#purr)\n  - [Intro](#intro)\n  - [Configuration](#configuration)\n  - [CLI](#cli)\n  - [Scheduled jobs](#scheduled-jobs)\n  - [REST API](#rest-api)\n  - [Writing checks](#writing-checks)\n  - [Development](#development)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Intro\n\nPURR (PUppeteer RunneR) is a devops-friendly tool for browser testing and monitoring.\n\nThe goal of this project is to have single set of browser checks, that could be used as tests, canaries in CI/CD pipelines and scenarios for production monitoring.\n\nThe tool uses puppeteer (\u003chttps://pptr.dev/\u003e) to run standalone browsers (Chrome and Firefox are supported currently).\n\nChecks results are stored as JSON reports, screenshots, traces and HAR files.\n\nPURR has three modes:\n\n- [CLI](README.md#cli) (mainly used in CI/CD pipelines)\n- [Queue worker](README.md#scheduled-jobs) (scheduled monitoring checks)\n- [REST service](README.md#rest-api) (show results and expose internal metrics for prometheus)\n\n## Configuration\n\n### data/checks dir\n\nStores descriptions of every single check\n\n### data/suites dir\n\nOrganizes checks into suites\n\n### data/parameters.yml\n\nSpecifies check parameters, i.e. target host or cookie values\n\n### data/schedules.yml\n\nDefine your schedules here\n\n### priority of parameters\n\n- Defaults from parameters.yml\n- Defaults from check/suite\n- Params from env\n- Explicitly specified params\n\n### PURR configuration\n\nYou can configure PURR behaviour using environmental variables. Please see the [ENV.md](./ENV.md) for details.\n\n## CLI\n\n### Requirements\n\n- docker\n- [docker compose](https://docs.docker.com/compose/install)\n- make\n\nBefore first run of whole application or custom checks, you need to provide `.env` file. Sample you can find in \n`.env.sample` file and full list of supported ENV variables in [ENV.md](./ENV.md).\n\n### Build\n\nNative docker build\n```shell\ndocker build -f $(pwd)/docker/Dockerfile -t ghcr.io/semrush/purr:latest .\n```\n\nOr use predefined make directive\n```shell\nmake docker-build\n```\n\n### Run single check\n\nNative docker run \n```shell\ndocker build -f $(pwd)/docker/Dockerfile -t ghcr.io/semrush/purr:latest .\ndocker run --rm -v $(pwd):/app --env-file $(pwd)/.env ghcr.io/semrush/purr:latest check example-com\n```\n\nOr use predefined make directive\n```shell\nmake run-check CHECK_NAME=example-com\n```\n\n### Run suite\n\nNative docker run\n```shell\ndocker build -f $(pwd)/docker/Dockerfile -t ghcr.io/semrush/purr:latest .\ndocker run --rm -v $(pwd):/app --env-file $(pwd)/.env ghcr.io/semrush/purr:latest suite example-com-suite\n```\n\nOr use predefined make directive\n```shell\nmake run-suite SUITE_NAME=example-com-suite\n```\n\n### Results\n\n```shell\n$ tree storage\nstorage\n├── console_log\n│   ├── console_semrush-com_0cedaca3-1153-47df-a616-55e21bf54635.log\n│   └── console_semrush-com_ded5990f-7638-48e6-9d0e-77f8dba376fd.log\n├── screenshots\n│   ├── screenshot_semrush-com_0cedaca3-1153-47df-a616-55e21bf54635.png\n│   └── screenshot_semrush-com_ded5990f-7638-48e6-9d0e-77f8dba376fd.png\n└── traces\n    ├── trace_semrush-com_0cedaca3-1153-47df-a616-55e21bf54635.json\n    └── trace_semrush-com_ded5990f-7638-48e6-9d0e-77f8dba376fd.json\n\n```\n\n### Traces and HARs\n\nPURR have a feature to save Chromium traces and [HARs](\u003chttps://en.wikipedia.org/wiki/HAR_(file_format)\u003e).\n\nYou can open traces in Chromium Devtools Network Inspector or [Chrome DevTools Timeline Viewer](https://chromedevtools.github.io/timeline-viewer/).\nFor HAR you can use [GSuite Toolbox HAR Analyze](https://toolbox.googleapps.com/apps/har_analyzer/).\n\n## Scheduled jobs\n\n### Run application\n\n```shell\ndocker compose up -d\n```\n\n### Apply schedules\n\n```shell\ndocker compose exec worker /app/src/cli.js schedule clean\ndocker compose exec worker /app/src/cli.js schedule apply\n```\n\n### Stop schedules\n\n```shell\ndocker compose exec worker /app/src/cli.js schedule clean\n```\n\n## REST API\n\nTo enable access to API server, just create file `docker-compose.override.yaml` and place replacement of `server` \nservice like in example:\n\n```yaml\nversion: '3.9'\n\nservices:\n  server:\n    ports:\n      - '8080:8080'\n```\n\nAfter that, all commands called via `docker compose` will apply configuration and provide access to server with address \n`http://localhost:8080`\n\n### Endpoints\n\n#### `GET /metrics`\n\nPrometheus metrics\n\n#### `GET /api/v1/checks`\n\nList of existing checks\n\n##### query strings\n\n#### `POST /api/v1/checks/:name`\n\nAdd check to queue\n\n##### Response\n\n**200**: Returns check report\n**202**: Returns id of created check job\n\n##### Payload\n\n- **name**: string\n  Check name to run\n- **params**: array\n  Any check parameter\n\n##### Query strings\n\n- **wait**: bool\n  **default**: false\n  Just return link for report when false\n- **view**: string\n  **default**: json\n  **options**: json, pretty\n  Output format\n\n##### Example:\n\n```shell\ncurl -X POST \\\n  -d 'params[TARGET_SCHEMA]=http' \\\n  -d 'params[TARGET_DOMAIN]=rc.example.com' \\\n  http://localhost:8080/api/v1/checks/main-page?wait=true\u0026view=pretty\n```\n\n#### `GET /api/v1/reports/:id`\n\nGet report\n\n##### Payload\n\n- **id**: string\n  Check report id\n\n##### Query strings\n\n- **view**: string\n  **default**: json\n  **options**: json, pretty\n  Output format\n\n#### `GET /api/v1/reports/:name/latest/failed`\n\nGet report\n\n##### Payload\n\n- **name**: string\n  Check report name\n\n##### Query strings\n\n- **schedule**: string\n  **default**: ''\n  Schedule name\n\n- **view**: string\n  **default**: json\n  **options**: json, pretty\n  Output format\n\n## Writing checks\n\nPURR translates scenario steps described in ./data/checks into methods of puppeteer.Page object.\nYou can check [puppeteer reference documentation](https://github.com/GoogleChrome/puppeteer/blob/v1.19.0/docs/api.md#class-page) for up-to-date capabilities.\n\n### Methods\n\nList of methods which were tested by the PURR dev team\n\n```yaml\n- goto:\n    - '{{ TARGET_SCHEMA }}://{{ TARGET_DOMAIN }}/{{ TARGET_PAGE }}/'\n\n- goto:\n    - '{{ TARGET_SCHEMA }}://{{ TARGET_DOMAIN }}/{{ TARGET_PAGE }}/'\n    - waitUntil: networkidle2\n\n- waitForNavigation:\n    - waitUntil: domcontentloaded\n\n- click:\n    - '{{ CSS_OR_DOM_SELECTOR }}'\n\n- type:\n    - '{{ CSS_OR_DOM_SELECTOR }}'\n    - '{{ STRING_TO_TYPE }}'\n\n- waitForSelector:\n    - '{{ CSS_OR_DOM_SELECTOR }}'\n\n- setCookie:\n    - name: '{{ COOKIE_NAME }}'\n      value: '{{ COOKIE_VALUE }}'\n      domain: .{{ TARGET_DOMAIN.split('.').slice(-2).join('.') }}\n```\n\n## Testing checks\n\nto launch your check run \n```\nmake check name=main-page\n```\n\n### Custom Methods\n\nCustom steps methods are described in [src/actions](./src/actions/common/index.js) dir and can be executed in checks.\n\n```yaml\n- actions.common.selectorContains:\n    - '[data-test=\"user-profile\"]'\n    - 'User Name:'\n```\n\n### Includes\n\nFeel free to use YAML anchors in your scenarios\n\n```yaml\n.login_via_popup: \u0026login_via_popup\n  - click:\n    - '[data-test=\"login\"]'\n  - waitForSelector:\n    - '[data-test=\"email\"]'\n  - type:\n    - '[data-test=\"email\"]'\n    - '{{ USER_EMAIL }}'\n  - type:\n    - '[data-test=\"password\"]'\n    - '{{ USER_PASSWORD }}'\n  - click:\n    - '[data-test=\"login-submit\"]'\n\n\nlogged-user-dashboard:\n  parameters:\n    USER_PASSWORD: secret\n  steps:\n    - goto:\n      - '{{ TARGET_URL }}'\n      - waitUntil: networkidle2\n    \u003c\u003c: *login_via_popup\n      parameters:\n        USER_EMAIL: root@localhost\n    - waitForSelector:\n      - '[data-test=\"user-profile\"]'\n    - actions.common.selectorContains:\n      - '[data-test=\"user-profile\"]'\n      - 'User Name:'\n```\n\n### Variables\n\nYou can specify parameters in checks and suites yaml files under 'parameters' key\n\n```yaml\nparameters:\n  TARGET_HOST: localhost\n\nvalid-password:\n  \u003c\u003c: *login_via_popup\n  parameters:\n    USER_EMAIL: root@localhost\n    USER_PASSOWRD: secret\n\ninvalid-password:\n  \u003c\u003c: *login_via_popup\n  parameters:\n    USER_PASSOWRD: invalid\n```\n\n### Proxy\n\nTo run a check, suite or schedule throw proxy use 'proxy' key\n\n```yaml\ncheck-page-from-india:\n  proxy: 'socks5h://user:password@proxy.service:8080'\n  steps:\n    - goto:\n        - '{{ TARGET_URL }}'\n    - waitForSelector:\n        - body\n    - actions.common.selectorContains:\n        - body\n        - 'Your location: India'\n```\n\n## Development\n\nMain entrypoint for project is `src/cli.js`.\n\nThere are two options for development avalaible.\n\n* cli command development require only call from cli. [docker-compose.single.yml](docker-compose.single.yml) placed for your convinience\n* client-server model. That mode described in [docker-compose.server.yml](docker-compose.server.yml). There we have two services avalaible\n  * sever - provides api endpoint and other stuff related to daemon itself\n  * worker - queue worker.\n\n\n```shell\nmake start-dev\nmake attach-dev\n```\n\n### Tests\n\nRun tests:\n\n```shell\nyarn run test\n```\n\n#### Mocks\n\nWe are using Jest testing framework.\n\nYou can mock module like that:\n\n```javascript\n// If `manual` mock exist in dir `__mocks__` along module file, will be used\n// automatically.\n//\n// Mocked module methods return `undefined`, fields return actual value.\njest.mock('../../config');\n```\n\n```javascript\n// Now `config` for all scripts will be `{ concurrency: 9 }`\njest.mock('../../config', () =\u003e ({ concurrency: 9 }));\n```\n\nOr like that:\n\n```javascript\nconst config = require('../../config');\n\nconfig.concurrency = 1;\nconfig.getWorkingPath = jest.fn().mockImplementation(() =\u003e {\n  return '/working/path';\n});\n```\n\n##### Be careful\n\nMethods `mock`\\\\`unmock` must be executed before module imports and in the\nsame scope.\nMocks state restoring after each test, but only when you did not used\n`jest.mock()`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemrush%2Fpurr","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsemrush%2Fpurr","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsemrush%2Fpurr/lists"}