{"id":14954822,"url":"https://github.com/wordpress/phpunit-test-runner","last_synced_at":"2025-04-09T05:12:20.771Z","repository":{"id":44894320,"uuid":"92320252","full_name":"WordPress/phpunit-test-runner","owner":"WordPress","description":null,"archived":false,"fork":false,"pushed_at":"2025-04-01T12:40:24.000Z","size":185,"stargazers_count":70,"open_issues_count":33,"forks_count":66,"subscribers_count":14,"default_branch":"master","last_synced_at":"2025-04-05T11:47:15.395Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"PHP","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/WordPress.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-24T17:49:09.000Z","updated_at":"2025-04-01T12:40:27.000Z","dependencies_parsed_at":"2024-03-17T17:28:44.655Z","dependency_job_id":"c9cd5c81-1a2f-48a3-a032-c42b944440ff","html_url":"https://github.com/WordPress/phpunit-test-runner","commit_stats":{"total_commits":198,"total_committers":27,"mean_commits":7.333333333333333,"dds":0.595959595959596,"last_synced_commit":"e91335b83907cff19d912f8d81469e1ab79afdee"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WordPress%2Fphpunit-test-runner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WordPress%2Fphpunit-test-runner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WordPress%2Fphpunit-test-runner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/WordPress%2Fphpunit-test-runner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/WordPress","download_url":"https://codeload.github.com/WordPress/phpunit-test-runner/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247980844,"owners_count":21027808,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:09:28.794Z","updated_at":"2025-04-09T05:12:20.755Z","avatar_url":"https://github.com/WordPress.png","language":"PHP","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PHPUnit Test Runner\n\nThanks for running the WordPress PHPUnit test suite on your infrastructure. We appreciate you helping to ensure WordPress’s compatibility for your users.\n\nIf you haven't already, [please first read through the \"Getting Started\" documentation](https://make.wordpress.org/hosting/handbook/tests/).\n\nThe test suite runner is designed to be used without any file modification. Configuration happens with a series of environment variables (see [.env.default](.env.default) for an annotated overview).\n\nAt a high level, the test suite runner:\n\n1. Prepares the test environment for the test suite.\n2. Runs the PHPUnit tests in the test environment.\n3. Reports the PHPUnit test results to WordPress.org\n4. Cleans up the test suite environment.\n\n## Setup\n\nThe test suite runner can be used in one of two ways:\n\n1. With GitHub Actions, (or Travis, Circle, or another CI service) as the controller that connects to the remote test environment.\n2. With the runner cloned to and run directly within the test environment.\n\nThe test runner is configured through environment variables, documented in [`.env.default`](.env.default). It shouldn't need any code modifications; in fact, please refrain from editing the scripts entirely, as it will make it easier to stay up to date.\n\nWith a direct Git clone, you can:\n\n```bash\n# Copy the default .env file.\ncp .env.default .env\n# Edit the .env file to define your variables.\nvim .env\n# Load your variables into scope.\nsource .env\n```\n\nIn a CI service, you can set these environment variables through the service's web console. Importantly, the `WPT_SSH_CONNECT` environment variable determines whether the test suite is run locally or against a remote environment.\n\nConcurrently run tests in the same environment by appending build ids to the test directory and table prefix:\n\n```bash\nexport WPT_TEST_DIR=wp-test-runner-$TRAVIS_BUILD_NUMBER\nexport WPT_TABLE_PREFIX=wptests_$TRAVIS_BUILD_NUMBER\\_\n```\n\nConnect to a remote environment over SSH by having the CI job provision the SSH key:\n\n```bash\n# 1. Create a SSH key pair for the controller to use\nssh-keygen -t rsa -b 4096 -C \"travis@travis-ci.org\"\n# 2. base64 encode the private key for use with the environment variable\ncat ~/.ssh/id_rsa | base64 --wrap=0\n# 3. Append id_rsa.pub to authorized_keys so the CI service can SSH in\ncat ~/.ssh/id_rsa.pub \u003e\u003e ~/.ssh/authorized_keys\n```\n\nUse a more complex SSH connection process by creating an SSH alias:\n\n```bash\n# 1. Add the following to ~/.ssh/config to create a 'wpt' alias\nHost wpt\n  Hostname 123.45.67.89\n  User wpt\n  Port 1234\n# 2. Use 'wpt' wherever you might normally use a SSH connection string\nssh wpt\n```\n\n## Running\n\nThe test suite runner is run in four steps. This explanation is for the local execution.\n\n### Requirements\n\nTo use the Runner, the following is required (testing WordPress 6.5):\n\n- Server / hosting (infrastructure) with the usual configuration you use\n- A database where you can test (tables will be created and destroyed several times)\n- PHP 7.0+ (view )\n- MySQL 5.5+ / MariaDB 10.0+\n- NodeJS 20.x / npm 10.x / grunt\n- PHP Composer\n- Git, RSync, WGet, UnZip\n\nTest environment:\n\n- Writable filesystem for the entire test directory (see [#40910](https://core.trac.wordpress.org/ticket/40910)).\n- Run with a non-root user, both for security and practical purposes (see [#44233](https://core.trac.wordpress.org/ticket/44233#comment:34)/[#46577](https://core.trac.wordpress.org/ticket/46577)).\n\n#### Database creation\n\n_This is an example for MySQL / MariaDB._\n\n```sql\nCREATE DATABASE wordpressdatabase CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci;\nGRANT ALL ON wordpressdatabase.* TO 'wordpressusername'@'localhost' IDENTIFIED BY 'wordpresspassword';\nGRANT ALL ON wordpressdatabase.* TO 'wordpressusername'@'127.0.0.1' IDENTIFIED BY 'wordpresspassword';\nFLUSH PRIVILEGES;\n```\n\n#### NodeJS installation\n\n_This is an example for Debian / Ubuntu._\n\n```bash\ncurl -fsSL https://deb.nodesource.com/setup_20.x | sudo -E bash -\nsudo apt -y install nodejs\nsudo npm install -g npm@latest\nnodejs --version\nnpm --version\n```\n\n#### PHP Composer\n\n_This is an example for Debian / Ubuntu._\n\n```bash\ncurl -sS https://getcomposer.org/installer -o composer-setup.php\nphp composer-setup.php --install-dir=/usr/local/bin --filename=composer\ncomposer --version\n```\n\n#### Git\n\n_This is an example for Debian / Ubuntu._\n\n```bash\napt -y install git\ngit --version\n```\n\n### Installing the Test Runner\n\nFirst, download the software. This example uses `/home/wptestrunner/` folder, but set the best for this environment.\n\n```bash\ncd /home/wptestrunner/\ngit clone https://github.com/WordPress/phpunit-test-runner.git\ncd phpunit-test-runner/\n```\n\nThe next step will be to configure the environment. To do this, make a copy of the example file and then configure it.\n\n```bash\ncp .env.default .env\nvim .env\n```\n\nThe content (in summary form) can be something like this:\n\n```bash\n###\n# Configuration environment variables used by the test runner\n#\n# # Create a copy for your local environment\n# $ cp .env.default .env\n#\n# # Make any necessary changes to the default values\n# $ vim .env\n#\n# # Load your variables into your environment\n# $ source .env\n###\n\n# Path to the directory where files can be prepared before being delivered to the environment.\nexport WPT_PREPARE_DIR=/tmp/wp-test-runner\n\n# Path to the directory where the WordPress develop checkout can be placed and tests can be run.\n# When running tests in the same environment, set WPT_TEST_DIR to WPT_PREPARE_DIR\nexport WPT_TEST_DIR=/tmp/wp-test-runner\n\n# API key to authenticate with the reporting service in 'username:password' format.\nexport WPT_REPORT_API_KEY=\n\n# (Optionally) define an alternate reporting URL\nexport WPT_REPORT_URL=\n\n# Credentials for a database that can be written to and reset.\n# WARNING!!! This database will be destroyed between tests. Only use safe database credentials.\n# Please note that you must escape _or_ refrain from using # as special character in your credentials.\nexport WPT_DB_NAME=\nexport WPT_DB_USER=\nexport WPT_DB_PASSWORD=\nexport WPT_DB_HOST=\n\n# (Optionally) set a custom table prefix to permit concurrency against the same database.\nexport WPT_TABLE_PREFIX=${WPT_TABLE_PREFIX-wptests_}\n\n# (Optionally) define the PHP executable to be called\nexport WPT_PHP_EXECUTABLE=${WPT_PHP_EXECUTABLE-php}\n\n# (Optionally) define the PHPUnit command execution call.\n# Use if `php phpunit.phar` can't be called directly for some reason.\nexport WPT_PHPUNIT_CMD=\n\n# (Optionally) define the command execution to remove the test directory\n# Use if `rm -r` can't be called directly for some reason.\nexport WPT_RM_TEST_DIR_CMD=\n\n# SSH connection string (can also be an alias).\n# Leave empty if tests are meant to run in the same environment.\nexport WPT_SSH_CONNECT=\n\n# Any options to be passed to the SSH connection\n# Defaults to '-o StrictHostKeyChecking=no'\nexport WPT_SSH_OPTIONS=\n\n# SSH private key, base64 encoded.\nexport WPT_SSH_PRIVATE_KEY_BASE64=\n\n# Output logging\n# Use 'verbose' to increase verbosity\nexport WPT_DEBUG=\n\n# Certificate validation\n# Use 1 to validate, and 0 to not validate\nexport WPT_CERTIFICATE_VALIDATION=1\n\n# WordPress flavor\n# 0 = WordPress (simple version)\n# 1 = WordPress Multisite\nexport WPT_FLAVOR=1\n\n# Extra tests (groups)\n# 0 = none\n# 1 = ajax\n# 2 = ms-files\n# 3 = external-http\nexport WPT_EXTRATESTS=0\n```\n\nConfigure the folder where the WordPress software downloads and the database accesses will be made in order to prepare the tests.\n\n### Preparing the environment\n\nBefore performing the first test, let’s update all the components. This process can be run before each test in this environment if wanted to keep it up to date, although it will depend more if it is in a production environment.\n\n```bash\ncd /home/wptestrunner/phpunit-test-runner/\ngit pull\nsource .env\ngit checkout master\n```\n\nIf you want to check a different branch, you can change it doing:\n\n```bash\ngit checkout example-branch\n```\n\n## Preparing the test\n\nNow there is the environment ready, run the test preparation.\n\n```bash\nphp prepare.php\n```\n\nThe system will run a long series of installations, configurations and compilations of different elements in order to prepare the test. If warnings and warnings come out you should not worry too much, as it is quite normal. At the end of the process it will warn you if it needs something it doesn’t have. If it works, you should see something like this at the end:\n\n```\nSuccess: Prepared environment.\n```\n\nNow that the environment has been prepared, the next step is to run the tests for the first time.\n\n### Running the test\n\nNow that the environment is ready, let’s run the tests. To do this, execute the file that will perform it.\n\n```bash\nphp test.php\n```\n\nWhat do the symbols mean?\n\n`.` → Each dot means that the test has been passed correctly.\n\n`S` → It means the test has been skipped. This is usually because these tests are only valid in certain configurations.\n\n`F` → Means that the test has failed. Information about why this happened is displayed at the end.\n\n`E` → It means that the test has failed due to a PHP error, which can be an error, warning or notice.\n\n`I` → Means that the test has been marked as incomplete.\n\nIf you follow these steps, everything should work perfectly and not make any mistakes. In case you get any error, it may be normal due to some missing adjustment or extension of PHP, among others. We recommend that you adjust the configuration until it works correctly. After all, this tool is to help you improve the optimal configuration for WordPress in that infrastructure.\n\n### Creating a report\n\nEven if the test has failed, a report will be made. The first one shows the information about our environment. Among the most important elements are the extensions that are commonly used in WordPress and some utilities that are also generally useful.\n\n```bash\ncat /tmp/wp-test-runner/tests/phpunit/build/logs/env.json\n```\n\nThe content of this file is somewhat similar to this:\n\n```bash\n{\n  \"php_version\": \"7.4.5\",\n  \"php_modules\": {\n    \"bcmath\": false,\n    \"curl\": \"7.4.5\",\n    \"filter\": \"7.4.5\",\n    \"gd\": false,\n    \"libsodium\": false,\n    \"mcrypt\": false,\n    \"mod_xml\": false,\n    \"mysqli\": \"7.4.5\",\n    \"imagick\": false,\n    \"pcre\": \"7.4.5\",\n    \"xml\": \"7.4.5\",\n    \"xmlreader\": \"7.4.5\",\n    \"zlib\": \"7.4.5\"\n  },\n  \"system_utils\": {\n    \"curl\": \"7.58.0 (x86_64-pc-linux-gnu) libcurl\\/7.58.0 OpenSSL\\/1.1.1g zlib\\/1.2.11 libidn2\\/2.3.0 libpsl\\/0.19.1 (+libidn2\\/2.0.4) nghttp2\\/1.30.0 librtmp\\/2.3\",\n    \"ghostscript\": \"\",\n    \"imagemagick\": false,\n    \"openssl\": \"1.1.1g 21 Apr 2020\"\n  },\n  \"mysql_version\": \"mysql Ver 15.1 Distrib 10.4.12-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2\",\n  \"os_name\": \"Linux\",\n  \"os_version\": \"4.15.0-20-generic\"\n}\n```\n\nIn addition to this report, a definitive file with all the information of what happened in the tests. This is the one that includes all the tests that are made (more than 10,000) giving information of the time that they take to be executed, problems that have arisen…\n\n```bash\ncat /tmp/wp-test-runner/tests/phpunit/build/logs/junit.xml\n```\n\nAt this point we can generate the reports by sending them to WordPress.org, if necessary. Even if you haven’t included the WordPress user (see below for how to create it), you can still run this file.\n\n```bash\nphp report.php\n```\n\n### Cleaning up the environment for other tests\n\nHaving the tests working, all that remains is to delete all the files that have been created so that we can start over. To do this, execute the following command:\n\n```bash\nphp cleanup.php\n```\n\n### Automatic running\n\nThe best way to run this test is to create a cron that runs everything. Having in mind that the tests can overlap, the best way can be using a systemd timer.\n\n```bash\ncat \u003e /etc/systemd/system/wordpressphpunittestrunner.service \u003c\u003c EOF\n[Unit]\nDescription=WordPress PHPUnit Test Runner\n[Service]\nType=oneshot\nExecStart=cd /home/wptestrunner/phpunit-test-runner/ \u0026\u0026 source .env \u0026\u0026 php prepare.php \u0026\u0026 php test.php \u0026\u0026 php report.php \u0026\u0026 php cleanup.php\nUser=wptestrunner\nGroup=wptestrunner\nEOF\n```\n\n```bash\ncat \u003e /etc/systemd/system/wordpressphpunittestrunner.timer \u003c\u003c EOF\n[Unit]\nDescription=WordPress PHPUnit Test Runner\n[Timer]\nOnCalendar=*-*-* *:*:00\nPersistent=true\n[Install]\nWantedBy=timers.target\nEOF\n```\n\n```bash\nsystemctl daemon-reload\nsystemctl enable wordpressphpunittestrunner.timer\nsystemctl start wordpressphpunittestrunner.timer\nsystemctl status wordpressphpunittestrunner.timer\n```\n\nIf you want to check how is everything working...\n\n```bash\njournalctl -u wordpressphpunittestrunner.timer\njournalctl -n 120 -u wordpressphpunittestrunner.service\n```\n\n## Contributing\n\nIf you have questions about the process or run into test failures along the way, please [open an issue in the project repository](https://github.com/WordPress/phpunit-test-runner/issues) and we’ll help diagnose/get the documentation updated. Alternatively, you can also pop into the `#hosting` channel on [WordPress.org Slack](https://make.wordpress.org/chat/) for help.\n\n## License\n\nSee [LICENSE](LICENSE) for project license.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwordpress%2Fphpunit-test-runner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwordpress%2Fphpunit-test-runner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwordpress%2Fphpunit-test-runner/lists"}