{"id":15954032,"url":"https://github.com/uhafner/autograding-model","last_synced_at":"2026-03-03T17:17:11.923Z","repository":{"id":37820906,"uuid":"263465886","full_name":"uhafner/autograding-model","owner":"uhafner","description":"Model to autograde Java projects based on a configurable set of metrics.","archived":false,"fork":false,"pushed_at":"2026-02-28T23:04:15.000Z","size":3422,"stargazers_count":5,"open_issues_count":0,"forks_count":9,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-01T11:47:26.090Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/uhafner.png","metadata":{"files":{"readme":"README.adoc","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2020-05-12T22:26:28.000Z","updated_at":"2026-02-28T23:04:19.000Z","dependencies_parsed_at":"2023-02-01T05:40:16.283Z","dependency_job_id":"8df95197-e53c-48f4-9a08-1087b79c34a9","html_url":"https://github.com/uhafner/autograding-model","commit_stats":null,"previous_names":[],"tags_count":100,"template":false,"template_full_name":null,"purl":"pkg:github/uhafner/autograding-model","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uhafner%2Fautograding-model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uhafner%2Fautograding-model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uhafner%2Fautograding-model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uhafner%2Fautograding-model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/uhafner","download_url":"https://codeload.github.com/uhafner/autograding-model/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/uhafner%2Fautograding-model/sbom","scorecard":{"id":603767,"data":{"date":"2025-08-11","repo":{"name":"github.com/uhafner/autograding-model","commit":"e4c8676bcb125085368d018ecb20fa98d26a28c9"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.8,"checks":[{"name":"Maintained","score":10,"reason":"30 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/13 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Dangerous-Workflow","score":0,"reason":"dangerous workflow patterns detected","details":["Warn: untrusted code checkout '${{ github.event.pull_request.merge_commit_sha }}': .github/workflows/quality-monitor-pit.yml:13","Warn: untrusted code checkout '${{ github.event.pull_request.head.sha }}': .github/workflows/quality-monitor-pit.yml:18"],"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: third-party GitHubAction not pinned by hash: .github/workflows/assign-pr.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/assign-pr.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/check-md-links.yml:10: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/check-md-links.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/check-md-links.yml:11: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/check-md-links.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:21: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/ci.yml:23: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/ci.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/ci.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/ci.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:26: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:29: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/codeql.yml:36: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:41: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:50: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/codeql.yml:57: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql.yml:65: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/codeql.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/coverage.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/coverage.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/coverage.yml:18: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/coverage.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/coverage.yml:25: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/coverage.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/coverage.yml:31: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/coverage.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/enforce-labels.yml:13: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/enforce-labels.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:19: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:31: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:35: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:45: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/quality-monitor-pit.yml:48: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/quality-monitor-pit.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/run-release-drafter.yml:14: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/run-release-drafter.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/sync-labels.yml:16: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/sync-labels.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/sync-labels.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/sync-labels.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/update-badges.yml:15: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/update-badges.yml:17: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:24: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:30: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:110: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:117: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:124: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:131: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:138: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Warn: third-party GitHubAction not pinned by hash: .github/workflows/update-badges.yml:154: update your workflow using https://app.stepsecurity.io/secureworkflow/uhafner/autograding-model/update-badges.yml/main?enable=pin","Info:   0 out of  17 GitHub-owned GitHubAction dependencies pinned","Info:   0 out of  21 third-party GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Token-Permissions","score":0,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql.yml:16","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql.yml:17","Warn: no topLevel permission defined: .github/workflows/assign-pr.yml:1","Warn: no topLevel permission defined: .github/workflows/check-md-links.yml:1","Warn: no topLevel permission defined: .github/workflows/ci.yml:1","Warn: no topLevel permission defined: .github/workflows/codeql.yml:1","Warn: no topLevel permission defined: .github/workflows/coverage.yml:1","Warn: no topLevel permission defined: .github/workflows/enforce-labels.yml:1","Warn: no topLevel permission defined: .github/workflows/quality-monitor-pit.yml:1","Warn: no topLevel permission defined: .github/workflows/run-release-drafter.yml:1","Warn: no topLevel permission defined: .github/workflows/sync-labels.yml:1","Warn: no topLevel permission defined: .github/workflows/update-badges.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"SAST","score":10,"reason":"SAST tool is run on all commits","details":["Info: SAST configuration detected: CodeQL","Info: all commits (19) are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}}]},"last_synced_at":"2025-08-21T01:07:31.929Z","repository_id":37820906,"created_at":"2025-08-21T01:07:31.929Z","updated_at":"2025-08-21T01:07:31.929Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30052328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T15:26:47.567Z","status":"ssl_error","status_checked_at":"2026-03-03T15:26:17.132Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-07T13:14:45.048Z","updated_at":"2026-03-03T17:17:11.904Z","avatar_url":"https://github.com/uhafner.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":":imagesdir: etc/images\n\n= Autograding Model\n\nimage:https://img.shields.io/badge/JDK-21-yellow.svg[JDK 21]\nimage:https://github.com/uhafner/autograding-model/workflows/GitHub%20CI/badge.svg[GitHub Actions, link=https://github.com/uhafner/autograding-model/actions/workflows/ci.yml]\nimage:https://github.com/uhafner/autograding-model/workflows/CodeQL/badge.svg[CodeQL, link=https://github.com/uhafner/autograding-model/actions/workflows/codeql.yml]\nimage:https://raw.githubusercontent.com/uhafner/autograding-model/main/badges/line-coverage.svg[Line Coverage,link=https://github.com/uhafner/autograding-model/actions/workflows/quality-monitor-comment.yml]\nimage:https://raw.githubusercontent.com/uhafner/autograding-model/main/badges/branch-coverage.svg[Branch Coverage,link=https://github.com/uhafner/autograding-model/actions/workflows/quality-monitor-comment.yml]\nimage:https://raw.githubusercontent.com/uhafner/autograding-model/main/badges/mutation-coverage.svg[Mutation Coverage,link=https://github.com/uhafner/autograding-model/actions/workflows/quality-monitor-comment.yml]\nimage:https://raw.githubusercontent.com/uhafner/autograding-model/main/badges/style.svg[Warnings,link=https://github.com/uhafner/autograding-model/actions/workflows/quality-monitor-comment.yml]\nimage:https://raw.githubusercontent.com/uhafner/autograding-model/main/badges/bugs.svg[Bugs,link=https://github.com/uhafner/autograding-model/actions/workflows/quality-monitor-comment.yml]\n\nimage::summary.png[Scores in Jenkins, width=\"100%\"]\nimage::comment.png[PR Comment, width=\"100%\"]\n\nThis Java \"Autograding Model\" library evaluates projects based on a configurable set of metrics:\n\n- Test statistics (e.g., number of failed tests)\n- Code coverage (e.g., percentage of covered lines)\n- Mutation coverage (e.g., percentage of survived mutations)\n- Static analysis warnings (e.g., number of SpotBugs warnings)\n- Software metrics (e.g., cyclomatic complexity)\n\nThe full list of supported metrics is defined by the parsers in the https://github.com/jenkinsci/analysis-model[analysis-model] and https://github.com/jenkinsci/coverage-model[coverage-model] libraries, which this library uses to read tool reports.\n\nThe autograding library reads reports produced by other tools (e.g., JUnit, JaCoCo, SpotBugs), aggregates the results, and evaluates them.\nDepending on the configuration, it computes a score to assess overall quality or generates a metrics report without scoring.\nIt is designed for CI environments such as Jenkins, GitHub Actions, and GitLab CI, and can annotate pull or merge requests.\nQuality gates can be enforced to fail a build if defined criteria are not met.\n\nThis autograding library is the foundation for the following tools:\n\n- https://github.com/uhafner/quality-monitor[GitHub quality monitor]: Monitors and enforces the quality of pull requests (or single commits) in GitHub.\n- https://github.com/uhafner/autograding-github-action[GitHub autograding action]: Computes an autograding score for student classroom projects in GitHub pull requests.\n- https://github.com/uhafner/autograding-gitlab-action[GitLab autograding action]: Computes an autograding score for student projects in GitLab merge requests.\n- https://plugins.jenkins.io/autograding/[Jenkins autograding plugin]: Shows the autograding results in Jenkins' UI.\n\n== Autograding Score Configuration\n\nWhen you use this library to compute an autograding score, you must define the metrics, their tools, and the scoring criteria in a JSON configuration.\nDetails about each metric are explained in the Metrics Documentation.\n\n.Example Autograding Configuration\n[%collapsible]\n====\n[source,json]\n----\n{\n  \"tests\": {\n    \"name\": \"JUnit Tests\",\n    \"id\": \"tests\",\n    \"tools\": [\n      {\n        \"id\": \"junit\",\n        \"name\": \"Unit Tests\",\n        \"pattern\": \"**/target/*-reports/TEST*.xml\"\n      }\n    ],\n    \"failureRateImpact\": -1,\n    \"maxScore\": 100\n  },\n  \"analysis\": [\n    {\n      \"name\": \"Style\",\n      \"id\": \"style\",\n      \"tools\": [\n        {\n          \"id\": \"checkstyle\",\n          \"pattern\": \"**/target/**checkstyle-result.xml\"\n        },\n        {\n          \"id\": \"pmd\",\n          \"pattern\": \"**/target/pmd-*/pmd.xml\"\n        }\n      ],\n      \"errorImpact\": -1,\n      \"highImpact\": -1,\n      \"normalImpact\": -1,\n      \"lowImpact\": -1,\n      \"maxScore\": 100\n    },\n    {\n      \"name\": \"Bugs\",\n      \"id\": \"bugs\",\n      \"icon\": \"bug\",\n      \"tools\": [\n        {\n          \"id\": \"spotbugs\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/spotbugsXml.xml\"\n        }\n      ],\n      \"errorImpact\": -3,\n      \"highImpact\": -3,\n      \"normalImpact\": -3,\n      \"lowImpact\": -3,\n      \"maxScore\": 100\n    }\n  ],\n  \"coverage\": [\n    {\n      \"name\": \"Code Coverage\",\n      \"tools\": [\n        {\n          \"id\": \"jacoco\",\n          \"name\": \"Line Coverage\",\n          \"metric\": \"line\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/site/jacoco/jacoco.xml\"\n        },\n        {\n          \"id\": \"jacoco\",\n          \"name\": \"Branch Coverage\",\n          \"metric\": \"branch\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/site/jacoco/jacoco.xml\"\n        }\n      ],\n      \"maxScore\": 100,\n      \"missedPercentageImpact\": -1\n    },\n    {\n      \"name\": \"Mutation Coverage\",\n      \"tools\": [\n        {\n          \"id\": \"pit\",\n          \"name\": \"Mutation Coverage\",\n          \"metric\": \"mutation\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/pit-reports/mutations.xml\"\n        },\n        {\n          \"id\": \"pit\",\n          \"name\": \"Test Strength\",\n          \"metric\": \"test-strength\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/pit-reports/mutations.xml\"\n        }\n      ],\n      \"maxScore\": 100,\n      \"missedPercentageImpact\": -1\n    }\n  ]\n}\n\n----\n====\n\n== Metric Report Configuration\n\nWhen you use this library to generate a metrics report without scores, you must define the individual metrics and their configuration in a JSON configuration, as shown below.\nThis configuration is a subset of the autograding score configuration, without scoring criteria.\nDetails about each metric are provided in the documentation at the end of this document.\n\n.Example Metric Configuration\n[%collapsible]\n====\n[source,json]\n----\n{\n  \"tests\": {\n    \"name\": \"Tests\",\n    \"tools\": [\n      {\n        \"id\": \"junit\",\n        \"name\": \"Unit Tests\",\n        \"pattern\": \"**/target/*-reports/TEST*util*.xml\"\n      },\n      {\n        \"id\": \"junit\",\n        \"icon\": \"no_entry\",\n        \"name\": \"Architecture Tests\",\n        \"pattern\": \"**/target/surefire-reports/TEST*archunit*.xml\"\n      }\n    ]\n  },\n  \"analysis\": [\n    {\n      \"name\": \"Style\",\n      \"id\": \"style\",\n      \"tools\": [\n        {\n          \"id\": \"checkstyle\",\n          \"pattern\": \"**/target/**checkstyle-result.xml\"\n        },\n        {\n          \"id\": \"pmd\",\n          \"pattern\": \"**/target/pmd-*/pmd.xml\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Bugs\",\n      \"id\": \"bugs\",\n      \"icon\": \"bug\",\n      \"tools\": [\n        {\n          \"id\": \"spotbugs\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/spotbugsXml.xml\"\n        },\n        {\n          \"id\": \"error-prone\",\n          \"pattern\": \"**/maven.log\"\n        }\n      ]\n    },\n    {\n      \"name\": \"API Problems\",\n      \"id\": \"api\",\n      \"icon\": \"no_entry_sign\",\n      \"tools\": [\n        {\n          \"id\": \"revapi\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/revapi-result.json\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Vulnerabilities\",\n      \"id\": \"vulnerabilities\",\n      \"icon\": \"shield\",\n      \"tools\": [\n        {\n          \"id\": \"owasp-dependency-check\",\n          \"icon\": \"shield\",\n          \"pattern\": \"**/target/dependency-check-report.json\"\n        }\n      ]\n    }\n  ],\n  \"coverage\": [\n    {\n      \"name\": \"Code Coverage\",\n      \"tools\": [\n        {\n          \"id\": \"jacoco\",\n          \"metric\": \"line\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/site/jacoco/jacoco.xml\"\n        },\n        {\n          \"id\": \"jacoco\",\n          \"metric\": \"branch\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/site/jacoco/jacoco.xml\"\n        }\n      ]\n    },\n    {\n      \"name\": \"Mutation Coverage\",\n      \"tools\": [\n        {\n          \"id\": \"pit\",\n          \"metric\": \"mutation\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/pit-reports/mutations.xml\"\n        },\n        {\n          \"id\": \"pit\",\n          \"metric\": \"test-strength\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/pit-reports/mutations.xml\"\n        }\n      ]\n    }\n  ],\n  \"metrics\":\n    {\n      \"name\": \"Software Metrics\",\n      \"tools\": [\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cyclomatic-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cognitive-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"npath-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"loc\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"ncss\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cohesion\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"weight-of-class\"\n        }\n      ]\n    }\n}\n----\n====\n\n== Quality Gates\n\nQuality gates enforce a minimum quality level for a project.\nFor example, you can require at least 80% line coverage and no critical bugs.\nThe following example shows how to define such a quality gate:\n\n.Example Quality Gate Configuration\n[source,json]\n----\n{\n  \"qualityGates\": [\n    {\n      \"metric\": \"line\",\n      \"threshold\": 80.0,\n      \"criticality\": \"FAILURE\"\n    },\n    {\n      \"metric\": \"spotbugs\",\n      \"threshold\": 0.0,\n      \"criticality\": \"UNSTABLE\"\n    }\n  ]\n}\n----\n\nTIP: The quality gate configuration is not part of the autograding score JSON configuration.\nIt is a separate configuration consumed by the corresponding tools.\nIn GitLab, pass the configuration via an environment variable; in GitHub Actions, pass it via an action input.\n\n== Report Scopes\n\nBy default, the autograding model analyzes the complete project.\nHowever, you can restrict the analysis to specific parts of the code base, e.g., only the files changed in a pull or merge request.\nThe following scopes are supported:\n\n- `ALL`: Analyze the complete project (default).\n- `CHANGED_FILES`: Analyze only files changed in the pull or merge request.\n- `NEW_CODE`: Analyze only code added or changed in the pull or merge request.\n\nYou can define the scope for each tool that participates in the autograding evaluation.\n\n.Example Quality Gate Configuration\n[source,json]\n----\n{\n  \"tools\": [\n    {\n      \"id\": \"pit\",\n      \"name\": \"Mutation Coverage\",\n      \"metric\": \"mutation\",\n      \"scope\": \"changed-files\",\n      \"sourcePath\": \"src/main/java\",\n      \"pattern\": \"**/mutations.xml\"\n    }\n  ]\n}\n----\n\nYou can also have a mix of different scopes for different tools.\nE.g., you can analyze the complete test results but only the coverage of changed files.\n\n== Metrics Documentation\n\nThe following sections describe each metric and its JSON configuration.\nEvery metric can be enabled and configured individually.\nAll configurations share the same structure: define a list of tools to collect the data, a name, an icon (Markdown emoji or https://openmoji.org/[OpenMoji]), and optionally a maximum score (if a score should be computed).\nEach tool must provide a https://docs.oracle.com/javase/tutorial/essential/io/fileOps.html#glob[glob pattern] that locates its result files in the workspace (e.g., JUnit XML reports) and a parser ID so the underlying model can select the correct parser.\nSee https://github.com/jenkinsci/analysis-model[analysis model] and https://github.com/jenkinsci/coverage-model[coverage model] for the list of supported parsers.\n\nOptionally, you can define the impact of each result (e.g., a failed test or a missed line) on the final score.\nImpacts are positive or negative numbers multiplied by the measured values during evaluation.\nWith negative impacts, the score starts at the maximum and subtracts the penalties.\nWith positive impacts, values add up towards the maximum score (capped at the maximum).\n\n=== Test statistics (e.g., number of failed tests)\n\nThis metric can be configured using a JSON object `tests`, see the following example:\n\n[source,json]\n----\n{\n  \"tests\": {\n    \"tools\": [\n      {\n        \"id\": \"junit\",\n        \"name\": \"Unittests\",\n        \"pattern\": \"**/junit*.xml\"\n      }\n    ],\n    \"name\": \"JUnit\",\n    \"passedImpact\": 10,\n    \"skippedImpact\": -1,\n    \"failureImpact\": -5,\n    \"maxScore\": 100\n  }\n}\n----\n\nYou can either count passed tests as positive impact or failed tests as negative impact (or use a mix of both).\nAlternatively, you can use the success or failure rate of the tests to compute the impact.\nThis alternative approach is shown in the next example:\n\n[source,json]\n----\n{\n  \"tests\": {\n    \"tools\": [\n      {\n        \"id\": \"junit\",\n        \"name\": \"Unittests\",\n        \"pattern\": \"**/junit*.xml\"\n      }\n    ],\n    \"name\": \"JUnit\",\n    \"successRateImpact\": 1,\n    \"failureRateImpact\": 0,\n    \"maxScore\": 100\n  }\n}\n----\n\nSkipped tests are listed individually.\nFor failed tests, the error message and stack trace are shown after the summary in the pull or merge request.\n\n=== Code or mutation coverage (e.g., line coverage percentage)\n\nThis metric can be configured using a JSON object `coverage`, see the following example:\n\n[source,json]\n----\n{\n  \"coverage\": [\n    {\n      \"tools\": [\n        {\n          \"id\": \"jacoco\",\n          \"name\": \"Line Coverage\",\n          \"metric\": \"line\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/jacoco.xml\"\n        },\n        {\n          \"id\": \"jacoco\",\n          \"name\": \"Branch Coverage\",\n          \"metric\": \"branch\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/jacoco.xml\"\n        }\n      ],\n      \"name\": \"JaCoCo\",\n      \"maxScore\": 100,\n      \"coveredPercentageImpact\": 1,\n      \"missedPercentageImpact\": -1\n    },\n    {\n      \"tools\": [\n        {\n          \"id\": \"pit\",\n          \"name\": \"Mutation Coverage\",\n          \"metric\": \"mutation\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/mutations.xml\"\n        }\n      ],\n      \"name\": \"PIT\",\n      \"maxScore\": 100,\n      \"coveredPercentageImpact\": 1,\n      \"missedPercentageImpact\": 0\n    }\n  ]\n}\n----\n\nYou can either use the covered percentage as positive impact or the missed percentage as negative impact (a mix of both makes little sense but would work as well).\nPlease make sure to define exactly a unique and https://github.com/jenkinsci/coverage-model[supported metric] for each tool.\nFor example, JaCoCo provides `line` and `branch` coverage, so you need to define two tools for JaCoCo.\nPIT provides mutation coverage, so you need to define a tool for PIT that uses the metric `mutation`.\n\nMissed lines or branches and survived mutations can be posted as comments in pull or merge requests if the corresponding tool supports this.\n\n=== Static analysis (e.g., number of warnings)\n\nThis metric can be configured using a JSON object `analysis`, see the following example:\n\n[source,json]\n----\n{\n  \"analysis\": [\n    {\n      \"name\": \"Style\",\n      \"id\": \"style\",\n      \"tools\": [\n        {\n          \"id\": \"checkstyle\",\n          \"name\": \"CheckStyle\",\n          \"pattern\": \"**/target/checkstyle-result.xml\"\n        },\n        {\n          \"id\": \"pmd\",\n          \"name\": \"PMD\",\n          \"pattern\": \"**/target/pmd.xml\"\n        }\n      ],\n      \"errorImpact\": 1,\n      \"highImpact\": 2,\n      \"normalImpact\": 3,\n      \"lowImpact\": 4,\n      \"maxScore\": 100\n    },\n    {\n      \"name\": \"Bugs\",\n      \"id\": \"bugs\",\n      \"icon\": \"bug\",\n      \"tools\": [\n        {\n          \"id\": \"spotbugs\",\n          \"name\": \"SpotBugs\",\n          \"sourcePath\": \"src/main/java\",\n          \"pattern\": \"**/target/spotbugsXml.xml\"\n        }\n      ],\n      \"errorImpact\": -11,\n      \"highImpact\": -12,\n      \"normalImpact\": -13,\n      \"lowImpact\": -14,\n      \"maxScore\": 100\n    }\n  ]\n}\n----\n\nTypically, negative impacts are used here so that each warning reduces the final score according to its severity.\nAll warnings can be posted as comments in pull or merge requests if supported.\n\n=== Software metrics (e.g., cyclomatic complexity)\n\nSoftware metrics can be configured using a JSON object `metrics`, see the following example:\n\n[source,json]\n----\n{\n  \"metrics\":\n    {\n      \"name\": \"Software Metrics\",\n      \"tools\": [\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cyclomatic-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cognitive-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"npath-complexity\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"loc\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"ncss\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"cohesion\"\n        },\n        {\n          \"id\": \"metrics\",\n          \"pattern\": \"**/metrics/pmd.xml\",\n          \"metric\": \"weight-of-class\"\n        }\n      ]\n    }\n}\n----\n\nCurrently, no impacts can be defined for software metrics; values are shown for reporting only.\nThis may change in the future, if there is a need for it.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuhafner%2Fautograding-model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fuhafner%2Fautograding-model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fuhafner%2Fautograding-model/lists"}