{"id":15282977,"url":"https://github.com/wikimedia/pixel","last_synced_at":"2026-02-28T10:01:45.117Z","repository":{"id":37048607,"uuid":"474465387","full_name":"wikimedia/pixel","owner":"wikimedia","description":"Catch visual regressions before users see them","archived":false,"fork":false,"pushed_at":"2025-05-13T19:09:53.000Z","size":1088,"stargazers_count":20,"open_issues_count":6,"forks_count":10,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-06-23T19:47:59.758Z","etag":null,"topics":["mediawiki","visual-diff","visual-regression-testing","wikipedia"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wikimedia.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-03-26T21:00:58.000Z","updated_at":"2025-06-14T12:35:26.000Z","dependencies_parsed_at":"2023-02-18T12:16:25.781Z","dependency_job_id":"ed4ff00b-e2c8-4dcc-b1fc-8d1ef7a7e697","html_url":"https://github.com/wikimedia/pixel","commit_stats":null,"previous_names":[],"tags_count":56,"template":false,"template_full_name":null,"purl":"pkg:github/wikimedia/pixel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2Fpixel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2Fpixel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2Fpixel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2Fpixel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wikimedia","download_url":"https://codeload.github.com/wikimedia/pixel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wikimedia%2Fpixel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29929582,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-28T09:58:13.507Z","status":"ssl_error","status_checked_at":"2026-02-28T09:57:57.047Z","response_time":90,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["mediawiki","visual-diff","visual-regression-testing","wikipedia"],"created_at":"2024-09-30T14:41:32.003Z","updated_at":"2026-02-28T10:01:45.095Z","avatar_url":"https://github.com/wikimedia.png","language":"JavaScript","readme":"[![Push or PR workflow](https://github.com/wikimedia/pixel/actions/workflows/push.yml/badge.svg)](https://github.com/wikimedia/pixel/actions/workflows/push.yml)\n\n# Pixel\n\n![Visual regression HTML reporter showing failing and passing tests](reporter.png)\n\n🚨 **Pixel is currently an experimental proof of concept working towards becoming a more stable and useful solution. Only use it when you're feeling dangerous.** 🚨\n\nPixel is a visual regression tool for MediaWiki developers/QA engineers that\nhelps you replace manual testing with automated tests that catch web ui\nregressions before users see them. It *currently* integrates\n[BackstopJS](https://github.com/garris/BackstopJS),\n[MediaWiki-Docker](https://www.mediawiki.org/wiki/MediaWiki-Docker), and Docker\nunder the hood.\n\nPixel can also be used for accessibility regression testing using [Pa11y](https://github.com/pa11y/pa11y) and [axe-core](https://github.com/dequelabs/axe-core) as the automated accessibility testing engine. Similar to visual regression testing, this makes it easier to see accessibility changes by comparing the results of accessibility tests between two patches.\n\nCheck out the hourly reports Pixel has generated at https://pixel.wmcloud.org\nwhich compare the latest release branch against master, and read\n[T302246](https://phabricator.wikimedia.org/T302246) for the motivation behind\nthe tool.\n\n## Quick Start\n\nFirst, clone the repo wherever you wish and `cd` into it:\n\n```sh\ngit clone https://github.com/wikimedia/pixel.git \u0026\u0026 cd pixel\n```\n\nPixel runs in multiple Docker containers to eliminate inconsistent rendering\nissues across environments. Please install\n[Docker](https://docs.docker.com/get-docker/) and \n**make sure it is running** prior to using Pixel.\n\nMake sure you are running a compatible version of Node.js. The required version of Node is noted\nin the [package.json](package.json#L14) file. We recommend using [NVM](https://github.com/nvm-sh/nvm#install--update-script) for managing different versions\nof Node.js.\n\nFinally, install the CLI dependency:\n\n```sh\nnpm install\n```\n\nRunning Pixel for the first time can take a long time (e.g. 5 minutes) as it\ndownloads Docker images, clones MediaWiki core (\u003e 2GB) and a number of\nextensions/skins. It will be much faster on subsequent runs as this data has\nalready been downloaded.\n\n## Usage\n\nYour workflow will usually involve the following ordered steps:\n\n### 1) Take reference (baseline) screenshots with `master` or release branch code\n\nIf you want to checkout the latest code in `master` or `main` from MediaWiki\ncore and all of its installed extensions and skins and then take reference\nscreenshots that your test screenshots (step 2) will be compared against, then:\n\n```sh\n./pixel.js reference\n\n```\n\nOr if you want the reference to be the latest release branch:\n\n```sh\n./pixel.js reference -b latest-release\n```\n\nOr if you want the reference to be a certain release branch:\n\n```sh\n./pixel.js reference -b origin/wmf/1.39.0-wmf.18\n```\n\nIf you want to run the mobile visual regression test suite pass the `--group\nmobile` flag.\n\n### 2) Take test screenshots with changed code\n\nIf you want to pull a change or multiple changes down from gerrit, take\nscreenshots with these changes on top of master and then compare these\nscreenshots against the reference screenshots, then\n\n```sh\n./pixel.js test -c Iff231a976c473217b0fa4da1aa9a8d1c2a1a19f2\n```\n\nNote that although change id `Iff231a976c473217b0fa4da1aa9a8d1c2a1a19f2` has a\n`Depends-On` dependency, it is the only change that needs to be passed. Pixel\nwill figure out and pull down the rest of the dependencies provided that it has\nthe relevant repositories (set in repositories.json).\n\nAn HTML report of your test results with screenshots will be opened\nautomatically on a Mac after the test completes. If you're not on a Mac, you can\nmanually open the file at `report/\u003cname-of-test-group\u003e/index.html`.\n\nAdditionally, Pixel runs a server at `http://localhost:3000` (default) which can\nbe used to interact with/debug the same server that the tests use.\n\nIf you want to run the mobile visual regression test suite pass the `--group\nmobile` flag.\n\n### 3) Accessibility regression testing\n\nPixel can also compare accessibility errors between a reference and test patch. In order to use this feature, just add the `--a11y` flag when running reference and test commands.\n\n```sh\n./pixel.js reference -b latest-release --a11y\n./pixel.js test -c Iff231a976c473217b0fa4da1aa9a8d1c2a1a19f2 --a11y\n```\n\nThe HTML report of the test results can be found at `report/\u003cname-of-test-group\u003e-a11y/index.html`.\n\n### Stopping the services\n\nIf you want to stop all of Pixel's services, run:\n\n```sh\n./stop.sh\n```\n\n### Rebuild\n\nStop and remove Docker containers, networks, and volumes and rebuild them:\n\n```sh\n./rebuild.sh\n```\n\nThis command will `git pull` the latest code from the `main` branch and destroy\nall Docker containers, volumes, and networks associated with Pixel. It will also\nrebuild any Docker images that need to be rebuilt to ensure that it is using the\nlatest database seed data and code.\n\n### Cleanup\n\nSometimes after making MediaWiki code changes, database changes, or having\nissues with the containers you just want to throw away everything and start\nPixel with a clean slate. To do that, run:\n\n```sh\n./clean.sh\n```\n\nYou can also reset Pixel's database to its original state with:\n\n```sh\n./reset-db.sh\n```\n\n## Development\n\n### Changing or adding tests\n\nAll tests are located in config files in the root directory (e.g.\nconfigDesktop.js, configMobile.js, configEcho.js, etc).\n\nFor example, to test a new url in the `mobile` group:\n\n1) Add a new object to the `tests` array in [configMobile.js](configMobile.js) with\nthe name of your test and its url path:\n\n```js\n{\n\tlabel: \"\u003cdescriptive-name-of-your-test\u003e\",\n\tpath: \"\u003curl-path-of-page\u003e\"\n}\n```\n\n2) Run the tests with:\n\n```sh\n./pixel.js reference -g mobile -b latest-release\n./pixel.js test -g mobile -b latest-release\n```\n\nThis should always pass if the tests are deterministic as it is testing the same\ncode in MediaWiki (no changes).\n\nFor more info on how to change or add test config, please refer to the\n[BackstopJS](https://github.com/garris/BackstopJS) README.\n\n#### Groups\n\nThere are several groups of tests that can be run:\n\n##### echo\n\nUI checks on the mobile and desktop site limited to the Echo (notifications feature).\n\n##### desktop\n\nUI checks on the desktop site.\n\n##### desktop-dev\n\nUI checks on the desktop site with various feature flags enabled. Feature flags must be manually\nupdated inside [configDesktopDev.js](configDesktopDev.js).\n\n##### mobile\n\nUI checks on the mobile site.\n\n\n### Configuring MediaWiki\n\nAll mediawiki config is in [LocalSettings.php](LocalSettings.php) and can be\nchanged. For example, perhaps you are working on a new feature in the `Vector`\nskin that is feature flagged and want to enable it.\n\n1) Edit [LocalSettings.php](LocalSettings.php)\n2) Check that your config is usable at `localhost:3000`\n3) Commit these changes and open a pull request in this repo with these changes\n\n### Updating the database\n\nPixel gets its seed data from MariaDB backups hosted at\nhttps://github.com/wikimedia/pixel-seed-data.\n\nPlease follow the [README.md](https://github.com/wikimedia/pixel-seed-data) in\nthat repo for step-by-step instructions on how to make database changes.\n\n### Adding new extensions or skins\n\nPixel ships with a number of MediaWiki extensions and skins already installed.\nPlease reference the [repositories.json](repositories.json) file to see a\nlist of these.\n\nTo add a skin or extension that isn't currently supported:\n\n1) Add it to the [repositories.json](repositories.json) file\n2) Make Pixel rebuild its code volume by running `./clean.sh`. This will\ndestroy every container, image, volume, network associated with Pixel so they\ncan be rebuilt. \n3) Configure the extension or skin in [LocalSettings.php](LocalSettings.php).\n4) Run `./pixel.js reference` to rebuild everything. This will also run the\n`maintenance/update.php` script which executes any needed database schema\nchanges. This will take awhile as Mediawiki core, extensions, and skins get\ndownloaded again.\n4) Check that your extension or skin is usable at `localhost:3000`\n5) If there were any database changes involved, you'll need to update the seed\ndata that is shared amongst users of Pixel. To do this, please follow steps 3\nthrough 9 on the [Pixel seed data](https://github.com/wikimedia/pixel-seed-data)\nrepo.\n6) Commit these changes and open a pull request in this repo with these changes\n\n## Issues\n\nPlease file all bugs, requests, and issues on the [web team's visual regression phabricator board](https://phabricator.wikimedia.org/project/board/5933/)\n\n## Ports\n\nIf you want to change the ports used you can modify the port env vars in the [.env](./.env) file\n\n`PIXEL_MW_HOST_PORT` is the port you can use to access pages from the host\n\n`PIXEL_NOVNC_PORT` is the port used by the NoVNC viewer when you use `WATCH_MODE=1`\n\nAfter changing them, run the `./rebuild.sh` script\n\n## Debugging\n\nThe following env vars are useful when running and debugging Pixel locally\n\n### Watch mode\n\nTo watch BackstopJS taking your screenshots (in its visual-regression container) you can use the `WATCH_MODE` env var:\n\n`WATCH_MODE=1 ./pixel.js reference`\n\nThis can be very helpful determining Mediawiki related configuration issues and should work with any command kicking off Pixel BackstopJS runs\n\n### Scenario filter\n\nTo quickly restrict which scenario runs without modifying any code, you can use the `SCENARIO_FILTER` env var:\n\n`SCENARIO_FILTER=\"Sticky header with unpinned TOC\" ./pixel.js reference`\n\nThis is useful when debugging specific scenarios and should work with any command kicking off Pixel BackstopJS runs\n\n### Scenario details\n\nTo see full scenario objects printed to the console during a run, you can use the `SCENARIO_DETAILS` env var:\n\n`SCENARIO_DETAILS=1 ./pixel.js reference`\n\nThis is also useful when debugging specific scenarios\n\n## Roadmap\n\nFuture plans can be seen by visiting the [web team's visual regression\nphabricator board](https://phabricator.wikimedia.org/project/board/5933/) under\nthe \"Roadmap (Epics)\" column.\n\n\n## Changelog\n\nFor more information about what's changed in each version, please refer to the [Changelog](./CHANGELOG.md)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikimedia%2Fpixel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwikimedia%2Fpixel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwikimedia%2Fpixel/lists"}