https://github.com/pmd/pmd-regression-tester
Automated regression tests against real-world projects
https://github.com/pmd/pmd-regression-tester
Last synced: 6 months ago
JSON representation
Automated regression tests against real-world projects
- Host: GitHub
- URL: https://github.com/pmd/pmd-regression-tester
- Owner: pmd
- License: bsd-2-clause
- Created: 2018-05-01T15:56:50.000Z (over 7 years ago)
- Default Branch: main
- Last Pushed: 2025-05-15T15:32:12.000Z (8 months ago)
- Last Synced: 2025-06-04T08:51:33.810Z (7 months ago)
- Language: Ruby
- Size: 1.17 MB
- Stars: 7
- Watchers: 9
- Forks: 5
- Open Issues: 2
-
Metadata Files:
- Readme: README.rdoc
- Changelog: History.md
- License: LICENSE
Awesome Lists containing this project
README
= pmdtester
home :: https://pmd.github.io
code :: https://github.com/pmd/pmd-regression-tester
bugs :: https://github.com/pmd/pmd-regression-tester/issues
build-status :: {
}[https://github.com/pmd/pmd-regression-tester/actions?query=workflow%3Abuild]
gem-version :: {
}[https://rubygems.org/gems/pmdtester]
== DESCRIPTION:
A regression testing tool ensure that new problems and unexpected behaviors will not be introduced to PMD project after fixing an issue , and new rules can work as expected.
== FEATURES/PROBLEMS:
The diff report can be generated according to the base and patch branch of PMD
on a list of standard projects(Spring Framework, Hibernate, Solr, etc.)
== SYNOPSIS:
=== Options:
-r, --local-git-repo path to the local PMD repository
-b, --base-branch name of the base branch in local PMD repository
-p, --patch-branch name of the patch branch in local PMD repository
-bc, --base-config path to the base PMD configuration file
-pc, --patch-config path to the patch PMD configuration file
-c, --config path to the base and patch PMD configuration file
-l, --list-of-project path to the file which contains the list of standard projects
-m, --mode the mode of the tool: 'local', 'online' or 'single'
single: Set this option to 'single' if your patch branch contains changes
for any option that can't work on main/base branch
online: Set this option to 'online' if you want to download
the PMD report of main/base branch rather than generating it locally
local: Default option is 'local', PMD reports for the base and patch branches are generated locally.
-t, --threads Sets the number of threads used by PMD. Set threads to 0 to disable multi-threading processing.
-f, --html-flag whether to not generate the html diff report in single mode
-a, --auto-gen-config whether to generate configurations automatically based on branch differences,this option only works in online and local mode
--filter-with-patch-config whether to use patch config to filter baseline result as if --auto-gen-config has been used. This option only works in online mode.
--keep-reports whether to keep old reports and skip running PMD again if possible
-d, --debug whether change log level to DEBUG to see more information
--error-recovery enable error recovery mode when executing PMD. Might help to analyze errors.
--baseline-download-url download url prefix from where to download the baseline in online mode
-v, --version
-h, --help
=== Quick start
==== Run local mode
pmdtester -b main -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -a
==== Run single mode
pmdtester -p YOUR_DEVELOPMENT_BRANCH -pc CONFIG_ONLY_CONTAINS_NEW_PMD_JAVA_RULE -m single
==== Run online mode
pmdtester -b main -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -m online -a
=== Output
The tool creates the following folders:
target
├── repositories <- the analyzed projects are cloned here
│ ├── PROJECT_NAME_1
│ ├── ......
│ └── PROJECT_NAME_n
├── reports
│ ├── BASE_BRANCH_NAME <- the base baseline is placed here
│ ├── PATCH_BRANCH_NAME <- the patch baseline is placed here
│ └── diff
│ ├── index.html <- the summary report of diff reports
│ ├── base_config.xml <- pmd config from the base branch
│ ├── patch_config.xml <- pmd config from the patch branch
│ ├── css <- css resources are placed here
│ ├── js <- js resources
│ ├── PROJECT_NAME_1
│ │ ├── project_data.js <- contains the violations as js/json
│ │ └── index.html <- the diff report of PROJECT_1
│ ├── .......
│ └── PROJECT_NAME_n
│ ├── project_data.js <- contains the violations as js/json
│ └── index.xml <- the diff report of PROJECT_N
├── pmd-bin--- <- cached pmd builds that are reused
└── pmd-bin-....
==== The baseline format
branch_name
├── branch_info.json
├── config.xml
├── STANDARD_PROJECT_NAME_1
│ ├── report_info.json
│ └── pmd_report.xml
├── ......................
│ ├── report_info.json
│ └── pmd_report.xml
└── STANDARD_PROJECT_NAME_n
├── report_info.info
└── pmd_report.xml
== REQUIREMENTS:
* Ruby 2.7 or higher
=== Runtime dependency
nokogiri ~> 1.13
slop ~> 4.6
differ ~> 0.1
rufus-scheduler ~> 3.8
logger-colors ~> 1.0
liquid ~> 5.2
=== Development dependency
hoe-bundler ~> 1.5
hoe-git ~> 1.6
minitest ~> 5.10
mocha ~> 1.5
rubocop ~> 0.93
test-unit ~> 3.5
rdoc ~> 6.4
== INSTALL:
gem install pmdtester --pre
== DEVELOPERS:
git clone https://github.com/pmd/pmd-regression-tester.git
cd pmd-regression-tester
gem install bundler
bundle config set path "vendor/cache"
bundle install # once
bundle exec rake verify # run this command before commit your changes
bundle exec pmdtester ... # run this to directly execute pmdtester from source
Run all unit tests:
bundle exec rake clean test
Run all integration tests:
bundle exec rake clean integration-test
Run a single test class, e.g.:
bundle exec ruby -I test test/test_project_diff_report.rb
bundle exec ruby -I test test/integration_test_runner.rb
Run a single test, e.g.:
bundle exec ruby -I test test/test_project_diff_report.rb -n test_diff_report_builder
=== Releasing
* Update +History.md+ (version and date)
* Update +lib/pmdtester.rb+ (version)
* Run "bundle exec rake verify" and add new +pmdtester.gemspec+ (new version)
* Commit ("Prepare release x.y.z").
* Tag this commit ("git tag vx.y.z").
* Update History.md and lib/pmdtester.rb for the next development version,
run again "bundle exec rake verify"
* Commit ("Prepare next development version x.y.z-SNAPSHOT").
* Push to main.
* Push the tag. Github Actions will build and publish the new gem
* A github release is automatically created, verify it on https://github.com/pmd/pmd-regression-tester/releases
* To make pmd's main CI use the new version (in [pmd/pmd](https://github.com/pmd/pmd/)), go to the root directory and run `bundle lock --update`. Commit these changes.
* Rename milestone to version, close it and create a new "Next" milestone: https://github.com/pmd/pmd-regression-tester/milestones